HamNava: A Dataset for Multi‑Label Instrument Classification

1.1 Iranian classical music

Though unique, the musical culture of Persia shares strong connections with the musical traditions of the Middle East and Central Asia. It also has ties to the music of the Indian subcontinent and, to a lesser extent, to African musical cultures and, in particular, from the 1800s onward, it shows influences from European music (Nettl, 2012). It is based on a system of ‘Dastgahs,’ which are a multi‑modal cycle or framework, consisting of a set of melodic modal structures organized in a cyclic pattern based on a modal foundation (Asadi, 2004). There are seven primary Dastgah, each encompassing distinct melodic patterns, motifs, and specific musical movements that evoke particular emotions.

The orchestration of Iranian classical music is mainly monophonic in solo performance and heterophonic when played in ensembles. Songs feature a single melodic line that is either sung or played, often accompanied by multiple instruments playing variations of it by adding ornamentation (Massoudieh, 2016). In contrast to polyphonic music, where multiple melodies occur simultaneously, Iranian classical music focuses on the ornamentation and expression of a single melody. Despite its monophonic nature, the performance typically involves a variety of instruments, each contributing to the music’s texture by interpreting the central melody in subtly different ways (Zaker Jafari, 2019). This approach adds depth and richness to the music while preserving its monophonic essence, presenting a challenge for recognizing individual instruments in this style compared to other polyphonic forms.

1.2 Our contribution

Our main contribution is the creation of a fully annotated multi‑instrument dataset with soft labels, designed not only for segment‑level multi‑label instrument classification in Iranian classical music but also for cross‑cultural research. This paper provides a detailed exploration of the dataset’s development process, including its crowd‑sourced annotation methodology. In summary, our contributions are threefold:

• Collection of an Iranian classical music, fully labeled, multi‑instrument dataset with crowd‑sourced soft labels, with a dynamic number of annotations;

• Introducion of an innovative crowd‑sourcing methodology that relies on social incentives and community involvement, rather than financial compensation for each annotation, using a common messaging platform; and

• establishment of a baseline for cross‑cultural multi‑label instrument classification as an MIR task through the use of pre‑trained music foundation models.

1.3 Related works

In this paper, we introduce HamNava, meaning ‘in unison’ or ‘harmonized’ in Farsi, a novel, crowd‑sourced dataset designed for multi‑label instrument classification in Iranian classical music as well as a baseline for evaluating models in cross‑cultural settings. The dataset provides 6,000 audio excerpts, each annotated for the presence or absence of nine core elements, including eight instruments and vocals, with a soft‑labeling approach that captures annotator confidence. Our methodology leveraged everyday messaging platforms to streamline the crowd‑sourcing process, relying on community engagement and social incentives to encourage participation.

Monophonic and polyphonic music instrument recognition are well‑established challenges in MIR, driving the creation of numerous datasets in this field. Table 1 provides an overview of widely used polyphonic music datasets and their key characteristics. These datasets are typically categorized as either partially labeled or fully labeled datasets according to the extent of their labeling.

Partially labeled multi‑instrument datasets feature labels for only a subset of the instruments present in each excerpt. This approach enables these datasets to encompass a wider variety of samples and a broader range of instruments. For example, some datasets may label only the predominant (most salient) instrument, e.g., IRMAS (Bosch et al., 2012), or they may include labels indicating the most or least likely instruments, e.g., OpenMIC‑2018 (Humphrey et al., 2018).

Fully labeled datasets, in contrast, typically provide both audio recordings and MIDI transcriptions, allowing for instrument identification at the note level. In some cases, the audio is synthesized to match the MIDI, e.g., Slakh2100 (Manilow et al., 2019) and Cerberus4 (Manilow et al., 2020), making these datasets less suitable for real‑world retrieval tasks. Other fully labeled datasets consist of live audio recordings, e.g., MedleyDB (Bittner et al., 2014), MusicNet (Thickstun et al., 2016), and URMP (Li et al., 2019). These real‑world datasets are generally smaller and less diverse, presenting challenges such as MIDI–audio alignment.

Several datasets focusing on musical traditions outside Western classical and popular music are increasingly used in computational musicology and the broader study of music through computational methods. The CompMusic Project (Serra, 2014) focuses on the computational study of five music traditions of the world; in addition to audio recordings, music scores, and lyrics, it emphasizes cultural aspect annotations. The Arab–Andalusian music dataset (Caro Repetto et al., 2018) includes annotations for Miznās, ub$\overline{u}$s, and Nawbas, capturing the rhythmic patterns and musical modes of this tradition. The Jingju corpus (Caro Repetto, 2018) contains more than 1,244 audio recordings across eight role types for the study of traditional Jingju singing; it is labeled for melodic analysis, with annotations for Shengqiang (modal systems) and Banshi (metrical patterns), and is complemented by Jingju score and lyrics collections. The OTMM dataset (Şentürk, 2016) contains more than 420 hours of stereo recordings in different Makams, Forms, and Usuls; as well as score collections in hand‑written scores and book formats. The Carnatic and Hindustani corpuses (Srinivasamurthy, 2016; Gulati, 2016) contain more than 800 hours of music recordings in different Rāgas, Tālas, and forms, providing a comprehensive foundation for research in Indian art music.

In addition, there are other newly proposed datasets that have not yet gained widespread recognition within the MIR community but have significant potential to advance cross‑cultural music research. These emerging datasets include collections focused on lesser‑explored traditions; one such example is the SAMBASET dataset (Maia et al., 2019), which documents historical and modern Brazilian samba de enredo recordings, or the Hardanger Fiddle Dataset (Lartillot et al., 2023), which contains annotated note and beat onsets for Norwegian Hardanger fiddle music. Another contribution is the Teach Yourself Georgian Folk Songs Teach Yourself Dataset (Gillman et al., 2022), an annotated collection of traditional vocal polyphony from Georgia, as well as the Erkomaishvili Dataset (Rosenzweig et al., 2020), which includes Georgian sacred three‑voice tracks with score–audio annotations. Similarly, the CCOM‑HuQin dataset (Zhang et al., 2022) features multimodal annotations for Chinese fiddle music, which records its playing techniques. CCMusic is a consolidated database of Chinese music for MIR research, containing datasets for instrument recognition, playing technique classification, mode classification, and singing style classification, open with standardized formats (Zhou et al., 2025). The Lyra dataset (Papaioannou et al., 2022) for Greek traditional and folk music captures tags of musical genre, instrumentation, and regional aspects, while the Choralbücher dataset (Gerhardt and Kirsch, 2024) offers a historical collection of hymn arrangements from Northern Germany with a focus on figured bass analysis. Annotated for beats and downbeats, the Brazilian Rhythmic Instruments Dataset (Maia et al., 2018) contains both solo‑ and multi‑instrument recordings across 10 instrument settings and five rhythm classes, and, similarly, the Candombe dataset (Nunes et al., 2015) features ensembles of three to five drums. In the context of Iranian classical music, the Heydarian mode‑recognition dataset (Heydarian and Bainbridge, 2019) includes 5,706 seconds of performances in five Dastgahs, played on the Santur. Additionally, the KDC dataset (Nikzat and Repetto, 2022) offers recordings of Ney, Santur, and voice across seven Dastgahs and five Avazes, totaling more than two hours of music, including performances and Radif s.

These datasets allow MIR models to capture regional, rhythmic, and modal subtleties across various musical traditions. However, they often face limitations due to a small number of instruments or excerpts, or they lack the diversity of the culture. This restricts their applicability to more diverse and extensive cross‑cultural MIR tasks.

2.1 Instrument selection

The selection of instruments for the HamNava dataset was carried out through a combination of expert consultation and empirical analysis. Professors specializing in Iranian classical music were consulted, and their expertise guided the selection of core instruments commonly featured in traditional ensembles. To complement this process, we conducted a survey of 300 albums spanning from 1975–2005, focusing on recordings by predominant Iranian artists. Naturally, we observed that the sound quality of older and live recordings tends to be lower than that of more recent releases, though never to an extent that hinders listening or annotation. The booklet and metadata of each album were examined to determine the specific instruments involved in the orchestration.

Through these processes, we consistently identified nine musical elements (eight instruments and a vocalist) across the majority of the albums. These elements include five string instruments (Tar, Setar, Santur, Oud, and Kamancheh), a wind instrument (Ney), two percussion instruments (Tonbak and Daf), and also voice. Our findings confirm that these nine elements establish a solid foundation for a balanced and representative dataset. This selection captures the diversity of instruments in Iranian classical music while remaining practical for full labeling and annotation.

The excerpts used for crowd‑sourcing were collected by investigating the aforementioned 300 albums, whose licenses could be secured for academic sharing. From these, we selected only the albums that featured the eight target instruments and voice, which reduced the number of albums to almost a quarter of the original set. Then, we extracted the tracks from these selected albums where two or more instruments were played together, resulting in 236 pieces. The instruments present in each track were identified collaboratively by the two musically‑trained authors, narrowing down the album‑level instrumentation to track‑level labels. These track‑level labels were later used to guide contributors during the annotation process for each five‑second excerpt extracted from each track.

Prior to extracting five‑second excerpts from the recordings, the tracks were categorized into two groups based on their complexity: easy and hard. The ‘easy’ category included tracks where all instruments were played with different techniques, e.g., plucking, striking, and bowing, making them more distinguishable for less‑experienced annotators. The ‘hard’ category, in contrast, contained excerpts that required a higher level of expertise for accurate annotation.

Ultimately, each track was segmented into non‑overlapping five‑second excerpts, resulting in over 17,000 candidates. To reduce this number while maintaining musical diversity, we applied a heuristic function that ranked and selected excerpts based on several criteria. First, the function ensured that each selected musical element appeared in at least 1,000 track‑level labels of excerpts, based on a narrowed‑down list of musical elements of the excerpts. Second, to promote diversity, no consecutive excerpts from the same track were selected. Third, to balance task difficulty, approximately twice as many excerpts were drawn from the ‘easy’ category compared to the ‘hard’ category. Finally, excerpts with the highest number of simultaneous musical elements were deprioritized, as such dense, heterophonic passages tend to be more challenging for accurate human annotation, deprived of visual reference.

This heuristic process resulted in a curated set of 6,000 five‑second excerpts for the HamNava dataset, with around 67% labeled ‘easy’ and the remaining ones labeled ‘hard’. The chosen balance between difficulty levels reflects the distribution of contributors who had joined the platform before the start of crowd‑sourcing. The final dataset size was determined by both the anticipated level of community engagement and the volume of available data. The five‑second duration was selected as a compromise, being long enough for annotators to make reliable judgments, while also being informative for models. This choice also aligns with recent practices in transformer‑based architectures and follows the current trend in foundation models for music. To preserve the authenticity of the source material, no post‑processing, such as silence removal or dynamic filtering, was applied to the excerpts, and each segment remains as it originally appeared in the licensed recordings.

2.2 Soft labeling and aggregation methodology

To efficiently gather reliable annotations for the HamNava dataset, a soft labeling approach was used. Soft labeling is especially beneficial in crowd‑sourcing environments as it enables faster annotations by lowering the threshold for precise decisions, leading to higher agreement rates among annotators (Martın‑Morató et al., 2023). Incorporating confidence elicitation into the annotation process, we aimed to collect probabilistic labels that capture the inherent ambiguity in the presence of instruments, especially in complex multi‑instrument contexts.

Annotators provided a confidence score for its presence using a simple confidence‑elicitation method with four discrete scales. These confidence scores were then averaged across annotators to generate soft aggregate labels. For vocals, however, we departed from the confidence scoring approach and directly asked annotators to specify whether the vocal contained Chah‑chah (also known as Tahrir, a rhythmic, wordless vocalization characterized by wide and rapid, vibrato‑like ornamentation over a fundamental note), Kalam (a melodic, with lyrics singing), or both. This categorical approach was chosen over confidence scoring because behavioral evidence suggests the human voice is special for humans (Liberman and Mattingly, 1989) and can be recognized more rapidly and accurately than instrumental sounds, due to neural mechanisms attuned to its complex spectro‑temporal features (Agus et al., 2012).

Eliciting confidence scores from annotators enhances label quality by encouraging them to consider their certainty. It also improves agreement in ambiguous cases and is cost‑effective, requiring fewer participants to achieve high‑quality annotations. These continuous labels help improve model training by incorporating uncertainty, reducing data requirements, and maintaining performance (Méndez Méndez et al., 2022). By using soft labeling and confidence elicitation, the accuracy and consistency of the annotations and the resulting labels for HamNava are significantly improved.

2.3 Crowd‑Sourcing platform and workflow

Our crowd‑sourcing session was conducted on Telegram, a widely‑used communication platform in Iran (Akbari and Gabdulhakov, 2019) and globally. We used Telegram’s bot feature, which enabled us to streamline and automate the crowd‑sourcing process for our dataset. The bot acted as a virtual assistant, guiding users through the annotation process without requiring manual intervention.

To maximize participation, the project was promoted within relevant Iranian classical music communities through advertisements and announcements on popular Telegram channels, encouraging users to engage and contribute. This outreach strategy aimed to attract individuals with an interest or background in Iranian classical music, motivating them to share their expertise based on the social dynamics of the dataset collection. The annotation project was launched on June 21, 2024, and ran for six months.

When users started the bot, they received a message outlining the process and requirements. To ensure high‑quality annotations and match users with appropriately leveled excerpts, each user’s ability to identify instruments was first evaluated. This assessment involved both self‑evaluation questions and practical instrument‑recognition tasks. Users could select their ear‑training level from ‘low,' ‘medium,’ and ‘high,’ each accompanied by a brief descriptive sentence. They then listened to a set of predefined musical sounds of Santur, staccato from Kamancheh, and a low‑quality Tar. Based on their responses, users were categorized into three proficiency levels according to their ability to distinguish instruments. Annotators who selected the correct answers and rated themselves the highest were assigned to the top category.

The lowest proficiency level was excluded from contributing to the dataset, as participants at this level generally lacked sufficient familiarity with the instruments. For the remaining two levels, the bot assigned specific types of audio excerpts: one group received excerpts containing multiple instruments playing the same technique, while the other group worked with simpler excerpts featuring clearer separations. This approach guaranteed that participants were assigned tasks suitable for their skill level, ultimately enhancing the quality of the annotations.

Once a user’s proficiency level is assessed, the annotation workflow begins. Annotators are provided with detailed instructions that ensure consistency in measurement, including clear definitions and distinctions between Chah‑chah and Kalam, as well as guidance on how to report their confidence levels. For each excerpt, annotators indicate whether they did not hear the instrument (confidence level 0), or, if they did hear it, they select a level from 1, 2, or 3 according to how confident they are in their judgment. These responses are later mapped to numerical values: 0.0 for ‘not heard,’ and 0.5, 0.75, and 1.0 for confidence levels 1, 2, and 3, equally spaced to reflect increasing levels of confidence in hearing the instruments. Each time the user enters the ‘\annotate’ command, a function is triggered that sends a randomly selected five‑second mono MP3 excerpt with a sample rate of 22,050 Hz, along with a predetermined list of potential instruments—based on the determined track‑level labels—for users’ focus listening. This follows a predefined sequence of questions, where each response reflects the user’s judgment about whether an instrument is absent or how confidently it is perceived to be present in the audio excerpt.

Throughout the annotations process, the total number of annotated excerpts was displayed to the user, which, in our pilot study, was shown to increase the average number of annotations per user. Users can pause or resume the annotation process at any time, offering flexibility in their participation. Every two weeks, participants received a message and a push notification (if enabled), reminding them about the process and stating the expected average number of annotations per user needed for dataset construction. The number of judgments collected for each excerpt was dynamically determined, and the annotation continued until two annotators agreed on the presence or absence of each instrument in the excerpt. Figure 1 shows the interface of the app and the annotation workflow.

Figure 1

3.1 Baseline model

For evaluations of HamNava, a data split of 70:15:15 was proposed for training, testing, and validation, while ensuring a consistent distribution of instruments and levels across all splits. The mean squared error (MSE) on the validation set was used to tune hyperparameters and select the best model configuration. Initially, a multi‑label regression task was performed, where both training and test samples were softly labeled, with performance assessed using MSE and $R^2$ metrics. Next, while maintaining the soft‑label regression for the training phase, the testing and validation tasks were framed as multi‑label binary classification with the threshold of $0.5$, with evaluation based on accuracy and F1 scores. Lastly, the labels of the training and testing sets were binarized, and accuracy and F1 metrics were reported for this setup.

In this section, we establish a performance baseline for multi‑label instrument classification in Iranian classical music. This baseline experiment helps to evaluate the effectiveness of simple classification and regression models before exploring more complex approaches, particularly those using pre‑trained models for cross‑cultural music information retrieval. By starting with a straightforward convolutional neural network (CNN) architecture, we can better understand the challenges and limitations of the task and provide a reference point for subsequent methods.

To establish a performance baseline for multi‑label instrument classification in Iranian classical music, we propose the following evaluation methodology using the HamNava dataset. These models serve as a reference point for evaluating more complex approaches. The CNN architecture consists of two convolutional layers, each followed by batch normalization, and four fully connected layers applied to the flattened feature representation. The ReLU activation function is employed after each layer. The network is optimized using the Adam optimizer with a learning rate of $0.001$ and a weight decay of $1e-4$. For the soft‑label setting, the loss function employed was the MSE, while, for the hard‑label setting, binary cross‑entropy was used. As input features, we extract Mel‑frequency cepstral coefficients (MFCCs) from the audio excerpts using the following configuration: FFT size of $1024$, hop length of $512$ samples, and number of Mel filterbanks of $40$. The results of the CNN baseline across the three tasks are summarized in Table 2.

While the CNN baseline provides a reasonable starting point, the shallow architecture may not fully capture the complex relationships between different instruments with relatively few samples, especially in the multi‑label case with high co‑occurrence patterns. This provides a reference point for subsequent methods, where we can better understand the challenges and limitations of the task. In the subsequent section, therefore, more advanced models, predominantly pre‑trained on Western classical and popular music, are explored to harness their effectiveness for cross‑cultural research.

3.2 Cross‑Cultural multi‑label instrument classification

Foundation models, such as large language and diffusion models, are versatile models designed to perform a wide range of tasks rather than being restricted to specific ones (Bommasani et al., 2021). These models are primarily trained through self‑supervision on large volumes of unlabeled data, allowing them to learn useful representations that can be applied effectively to various downstream tasks (Ericsson et al., 2022).

In the domain of music processing, foundation models offer the potential to reduce annotation costs and enhance generalization. Their use could significantly improve music information retrieval and generation for various music‑related tasks (Ma et al., 2024). While these models show promise for advancing cross‑cultural and culture‑preservation works (Ma et al., 2024), research on foundation models pre‑trained on large datasets, which are typically dominated by Western or Western‑influenced music, remains limited in the context of non‑Western and folk music (BabaAli and Mohseni, 2024).

This subsection examines the cross‑cultural potential of foundation models on HamNava, highlighting notable state‑of‑the‑art models from the literature, such as Music2vec (Li et al., 2022), MusicHuBERT (Ma et al., 2023), and MERT (Li et al., 2023). Music2vec, inspired by Data2vec (Baevski et al., 2022), is an efficient model for learning representations from raw music waveforms through a teacher–student architecture with masked input prediction. MusicHuBERT adapts the HuBERT (Hsu et al., 2021) framework to music by predicting discrete pseudo‑labels for masked audio segments, using pre‑trained K‑means clustering on MFCC features, which contrasts with Music2vec’s continuous latent predictions. MERT, on the other hand, integrates acoustic and musical representation learning by combining RVQ‑VAE (Défossez et al., 2022) and CQT‑based teacher models in a multi‑task framework.

Considering the previous evaluation settings, a fully connected decision layer was added to each of the three models, with all layers except the final one frozen during training. The learning rate was set to $0.01$, with 100 epochs and the Adam optimizer used for training. Table 2 summarizes the results, showing Music2vec’s superior performance across all settings. Using p‑values to assess statistical significance, we find that the pre‑trained Music2vec model performs significantly better in representing excerpts containing folk elements than the CNN‑based architecture ($p\text{-value}<0.005$). While this improvement is not statistically significant when comparing Music2vec with MERT ($p\text{-value}>0.1$), representations learned by the MusicHuBERT model fail to generalize to the selected musical elements in Iranian classical music, given these evaluations and as reflected by its lower performance.

Additionally, Table 3 presents the instrument‑wise metric values for the best‑performing model. Among all the musical elements, the singer consistently achieved the highest performance across almost all settings, which may be due to the presence of vocal samples in the training data of the foundation models. Tonbak (percussion) and Oud (string) also ranked among the top‑performing instruments after ‘singer.’