Seminar: Semantic-Driven Music Score Generation

Note: This page (and the content on it) is work in progress until the start of the lecture period.

This seminar explores how semantic structures extracted from text can control the generation of symbolic musical scores. In phase 1 (April 16th-May 18th) Students build a minimal semantic-driven music score generation system using LLM-based text representations and Markov chains (e.g. mapping machine-learning–based semantic models or LLM-based transformer embedded vectors to pitch) using Python and LilyPond. Semantically related concepts (“sword” / “dagger”) should result in related musical material, while distant concepts (“sword” / “airport”) should sound correspondingly different. These systems will be part of a public live lecture concert at Musikfestspiele Saar. Important: As the first system will be part of a public concert, it is important to hold the deadline (see below) in Phase 1. 

In phase 2 (May 28th-June 25th) students each analyse and compare one existing AI–music system or research paper, using their own implementations and The (Un)Answered Question as benchmarks.

In the final phase 3 (June 25th-August 27th) students redesign their approach and develop a more advanced semantic-driven system focusing on musical structure, polyphony, and motivic development. At the end of semester submission of code and a video documentation (10-15min) about the advanced system is due as the final output. 

The seminar combines hands-on programming, experimental composition for a live performance, and research-oriented reflection. It is aimed at students interested in AI, creative systems, and symbolic representation of music.

Requirements

  • Basic programming experience (Python)
  • Basic knowledge of or willingness to gain knowledge in AI, machine learning, or creative systems
  • No prior music theory knowledge required, ability to read a music score is helpful. Willingness to work with LilyPond (https://lilypond.org/)
  • Willingness to work collaboratively and experiment
  • Willingness to attend the Lecture Concert (VHS Building near Schloss Saarbrücken) - attending this is mandatory for getting a certificate
  • Availability in lecture free period until end of August (finalization of system, video documentation)
  • Reliability and dependability

Contact

Martin Hennecke

Dates

Regular meeting time: Thursdays, 14:15–15:45 / 2:15pm-3:45pm.

Nr. Date Content
00 09.04.26 No meeting (seminars assignment will happen on this day)
01 16.04.26 Kickoff - Session 0 - will be done remotely; link will be shared with participants via email
02 23.04.26 Session 1 (@DFKI, Reuse)
03 30.04.26 Session 2 (@DFKI, Barwise)
04 07.05.26 Session 3 (@DFKI, Reuse)
D 11.05.26
(Monday!)		 22:00 / 10pm: Submission deadline for sprint 1 system (via email)
  14.05.26 Public holiday; no meeting
05 18.05.26
(Monday!)		 20:00 - 22:00 / 8-10pm: Lecture Concert (Musikfestspiele Saar at VHS-Building near Schloss Saarbrücken; ticket will be provided); Session 4 - Attendance mandatory
  21.05.26 No meeting (time for writing the concert report)
D 26.05.26
(Tuesday!)		 22:00 / 10pm: Submission deadline for concert report (via email)
06 28.05.26 Session 5 (@DFKI, Barwise)
  04.06.26 Public holiday; no meeting (individual reading & preparation)
  11.06.26 No meeting (individual reading & preparation)
07 18.06.26 Session 6 (paper presentations part 1) (@DFKI, Reuse)
08 25.06.26 Session 7 (paper presentations part 2) (@DFKI, Barwise)
09 02.07.26 Session 8 (@DFKI, Barwise)
  09.07.26 no meeting (individual preparation)
10 16.07.26 Session 9 (@DFKI, Barwise)
     
D 27.08.26 22:00 / 10pm: Final submission and video documentation hand-in


Location

  • Session 0: online, link will be provided via email.
  • Session 1,3,7: DFKI Saarbrücken, room Reuse (main building: -2.17). Enter DFKI through the main entrance, walk past the elevator in the foyer on your right, and go through the double doors on the right. 
  • Session 2, 5, 8, 9: DFKI Saarbrücken, room Barwise (main building: -2.07) - it's behind the DFKI reception.
  • Session 4: Musikfestspiele Saar at VHS-Building near Schloss Saarbrücken


Assessment Criteria

See structure overview below.

Phase 1 (35%)

  • Functional semantic-driven score generation system
  • Clear mapping from semantic representation to musical parameters
  • Quality of explanation and reflection
  • Concert report depth and critical analysis

Phase 2 (25%)

  • Quality of paper analysis
  • Understanding of system architecture and agency
  • Ability to compare systems critically
  • Clarity of presentation

Phase 3 (40%)

  • Conceptual quality of redesigned system 
  • Integration of structure, polyphony, and motivic logic
  • Appropriate use of AI methods (not necessarily complexity)
  • Video documentation

Attending session 4 (the concert) is a mandatory requirement for passing the course in general.


Structure Overview

Phase 1 – Sprint 1: Semantic Text → Music (Session 1-4)

Overall goal of Phase 1
Students develop a first, intentionally simple semantic-driven score generation system with Python and LilyPond (monophonic). This will be potentially showcased at the lecture concert.
The focus lies on transparency, explainability, and symbolic representation.

Semantic information (e.g. similarity between words, sentences, or sections) is mapped to local musical decisions, primarily through a Markov-based process. The output is a score pdf and midi via LilyPond. 

The emphasis is not on musical quality, but on making semantic relations audible and visible.

The group decides which and how many of the developed systems are used in the real performance situation.

The generated scores will be performed by a self-playing piano in combination with live performers as part of the lecture concert on May 18th.

All students are required to attend the concert and submit a short reflective review.

Deliverables: During phase 1, each student submits:

1) Sprint 1 System 

  • Monophonic semantic-driven music score generator
  • Python implementation (semantic analysis + Markov-based decision layer)
  • LilyPond output (PDF score + MIDI)

2) System Documentation

  • Short written description explaining:
    • text input and semantic representation,
    • mapping from semantics to musical parameters, describing how LLM is used,
    • role of the Markov chain.

3) Lecture Concert Contribution

  • Active participation 
  • Analytical report addressing:
    • what are the advantages in using AI in a classical concert
    • how was the audience perception on „art meets sciences“,
    • to what extent adaptive or learning-based AI methods were used. 

Phase 2 – Theory & Comparison (Sessions 5–7)

Overall goal of Phase 2
Students analyse existing AI–music systems and research papers as design references, using their own Sprint-1 systems and The (Un)Answered Question as benchmarks.
The focus lies on identifying where intelligence actually operates within these systems, which decisions are automated, and which forms of intelligence are absent or simulated.

Deliverables:

During phase 2, each student submits:

1) Paper / System Presentation

  • Oral presentation of one selected AI–music system or research paper
  • Focus on:
    • system architecture
    • distribution of agency (AI vs. human)
    • representation (symbolic, audio, hybrid)
    • The (Un)Answered Question vs. the selected system
    • the student’s own Sprint-1 system as an additional reference
  • Critical assessment of:
    • “how much AI” is actually present
    • where algorithmic logic ends and design decisions begin

2) Design Requirements List

  • Short, concrete list of requirements for a more advanced system (of sprint 1 system)

Phase 3 – Sprint 2: More advanced semantic-driven music score generation system (Sessions 8–9)

Overall goal of Phase 3
Students redesign their systems.

The goal is to move:

  • from single notes to motivic development
  • from local similarity to musical identity and memory
  • from monophonic output to structured polyphony

Semantic similarity is now explored at the motivic and structural level: The semantics of a text should steer the dramaturgy of a whole piece of music.

In addition to melodic parameters, further musical dimensions such as harmony or texture may be explored.

Deliverables: 

During phase 3, each student submits:

1) Sprint 2 System

  • Polyphonic / motivic prototype
  • Use of different LLM-based semantic representations (e.g. transformer embeddings),
  • System Architecture Documentation
  • Diagram and short explanation of:
    • semantic representation(s),
    • motivic logic / musical decision layers.
    • which LLM-based semantic representation works best (and why)

2) final code + video documentation