COMP3717 Introduction to Music Processing - Summative Assignment: Musical output
Goal
Your objective is to develop an algorithmic system that can produce novel musical output in one of the following formats:
-
- MIDI
-
- MusicXML
-
- Audio (.wav)
-
- Video
You will present your algorithmic system by submitting a demonstration video showing the system in operation, and by describing its operation in a 1,000-word technical note (“report”). You will additionally submit the code that specifies the implementation shown in the demonstration video. Before the final submission, you will also produce one-page outline of the report, which should show evidence of concrete progress toward the final submission.
Requirements
Your system must involve some combination of at least two of the following techniques:
-
1) reading, splitting, and recombining audio files
-
2) Fourier transform and/or inverse Fourier transform
-
3) pitch-class profiles
-
4) rhythmic prototypes
-
5) sequential models of music data (harmony, melody, and/or rhythm)
-
6) hierarchical models of music data
-
7) neural models of music data
This system must be implemented in Python >= 3.8. You are welcome to use the NCC JupyterHub instance for this module to develop your solution.
Your system must process, be trained on, or otherwise consume audio and/or symbolic data from one of the following datasets:
COMP3717 - Epiphany 2024 - Summative Coursework Specification v2024-01-24 2
-
Hawthorne, Curtis, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, and Douglas Eck. “Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset.” Presented at International Conference on Learning Representations (2019). https://doi.org/10.48550/arXiv.1810.12247.
-
- Download v. 3.0.0 from https://magenta.tensorflow.org/datasets/maestro
-
- We will make available a fixed subset of the MAESTRO dataset for
development purposes.
-
-
Zalkow, Frank, Stefan Balke, Vlora Arifi-Müller, and Meinard Müller. “MTD: A
Multimodal Dataset of Musical Themes for MIR Research.” Transactions of the International Society for Music Information Retrieval (TISMIR) 3, no. 1 (2020): 180– 92. https://doi.org/10.5334/tismir.68.
You do not need to use the whole dataset you have chosen. You may use other datasets or music data, not listed here, in addition to - but not instead of - this dataset. You should aim for a certain amount of creativity in your output, but the main objective of this coursework is to showcase your understanding of your chosen techniques.
Summary of required materials
-
- Stage 1 - Report outline (20%)
o One.pdffile,namedoutline_abcd89.pdf(1page) -
- Stage 2 - Final submission (80%)
o A single .zip file, named final_abcd89.zip, comprising:
▪ Final report.pdf file (1,000 words)
▪ Demonstration video (maximum 3-minute video file)
▪ Code
-
main.py or main.ipynb
-
Any other files, as required (see below)
Where “abcd89” is your CIS username.
Stage 1 - Report outline
Deadline: 19/03/2024 at 14:00
Your report outline must offer a bullet-point outline of your final report, dealing with each of the required headings (see below) in turn. Your report outline must not exceed 1 (A4) page. Your report outline must be submitted in .pdf format.
The outline serves as a written plan for the content of the final report and should be concise. Your outline should clearly show progress towards the demonstration video and the technical report. You do not need provide a references section at this stage.
Stage 2 - Final submission
Deadline: 30/04/2024 at 14:00
All elements of the submission (the report and the video) should evidence a thorough understanding of the theoretical and practical aspects of at least two of the techniques chosen. This will be reflected in the accurate and precise use of technical terminology, notation, figures, and programming conventions (as appropriate) and their use will be consistent with the usage in lecture and practical. It is also generally expected that you illustrate your explanations with reference to the code and/or video, where relevant.
COMP3717 - Epiphany 2024 - Summative Coursework Specification v2024-01-24 3
Final report
Your final report should concisely and precisely describe the operation of your system and must have the following structure:
-
Introduction (c. 200 words)
Briefly introduce and explain the concept(s) underlying your algorithmic system. State the two (or more) techniques from the given list that your system uses. Describe the dataset(s) that your system uses to produce an output. Explain any pre- or post- processing steps that are used.
-
Approach (c. 500 words)
Describe in sufficient detail the two (or more) chosen techniques used in your system, with the appropriate use of formulae and/or diagrams, where appropriate. Make reference to your code (e.g., by filename and line number) and demonstration video (e.g. by timestamp), where appropriate. Mention any inspirations or models for your system in the scientific literature or any other prior art (e.g., existing commercial or creative systems).
-
Evaluation (c. 300 words)
Describe any techniques you have used to assess the output your system produces, which could include quantitative, qualitative, or mixed methods. Report the results these techniques produce. Reflect on the success or failure of the approaches you adopted. Describe any possible future extensions of this work.
You must also include a references section, which will not count against the word limit, which is a hard maximum of 1,000 words. Your final report must be in .pdf format.
You should use a single-column, technical report format, using (e.g.) LaTeX or Word/LibreOffice.1 Citations and references should be provided in a style that is well-defined and appropriate to the field (e.g., IEEE). You should include formulae as well as properly captioned and attributed figures, where relevant, to illustrate your report.
Demonstration video
You must submit a video that clearly demonstrates your Python code in use, to produce a typical output of your system. You must ensure that any source code you show in this is readable. If your system produces audible output or consumes audible input (e.g., a visualiser), your video must include this audio. The video should be long enough to demonstrate the operation of your system and must not exceed three (3) minutes. The demonstration video must be playable using VLC Media Player on a standard CIS desktop.
This may be produced using screencast software, such as OBS Studio.2
Code
You must also submit the same Python code used to produce the working system that you present in the demonstration video. The code may be consulted by markers to validate references to the code made in the report and/or depicted in the demonstration video.
Your submitted code should include a single main.ipynb or main.py file that can be executed and reproduces the output file you provide. This may depend on other files, standard library modules, and/or third-party modules. You should provide a README.md file that explains
how the code should be used, and how to meet any required dependencies (e.g., on third- party modules or codecs), for example by using a requirements.txt file or some other means.
If your system requires data from datasets other than the ones mentioned above at run time, please ensure that these are provided as part of your submission. If your system requires some long-running step before producing output (e.g., training a model), please include a pre-trained model (checkpoint, *.pkl file or similar) and load that in your code. You should also include the code required for training such models from scratch.
Marking criteria
Stage 1 - Report outline (20%)
Your outline will be marked out of 20, in line with the departmental assessment criteria3 and the following rubric:
Marks Criteria
20 The outline demonstrates concrete progress toward the video submission and the
final report, under each of the required headings.
You will also receive feedback on your outline, identifying strengths and areas for
improvement in advance of your final submission.
Stage 2 - Final submission (80%)
Your final submission will be marked out of 80, in line with the departmental assessment
criteria and the following rubric:
