The goal of RagaPatch is to create a computer program that can interactively learn to improvise melodies in a raga-like manner in response to another player or by itself. RagaPatch is written in MAX, Opcode's graphical programming environment for music applications.
The first step is to understand the melodic movement of a raga and to be able to translate that understanding into rules that could become the basis for such a program.
In order to understand the melodic movement of a raga one must first consider the traditional ways that these possibilities are codified and transmitted. These traditional ways of raga codification can be thought of as existing in layers. Layer one is the simple ascending and descending patterns called respectively arohi and avarohi which will often include the pakad or catch phrase that will evoke the raga and will distinguish it from all other ragas. The next layer, more detailed, is the chalan or "road map" that shows more possibilities than the simple ascending or descending pattern and will also include the pakad or catch phrase. Bandish, or fixed composition (including sargam, songs & instrumental compositions), a still more detailed layer, shows the possibilities of a raga in conjunction with words or rhythmic Bol patterns, in the context of meter and meaning. At this layer one encounters traditional rhythmic phrases and practices. The deeper, more detailed layers of knowledge come from exposure to a master musician, who can demonstrate nuances of pitch, (swara and sruti), ornamentation, (meend, gamak, zamzama, alankar, etc.), and rhythm (tala, laya, chand), all in the context of the proper mood and exposition of a raga, and from one's own practice which includes performance as well as listening to great masters of the art.
Since the deeper layers of raga are beyond the scope of this work, I will concentrate on the first three layers: 1) ascending/descending, 2) chalan & pakad, 3) fixed composition.
All of the following examples will be illustrated in both sargam (Indian solfeggio) and western staff notation. For a full explanation of sargam and all other Indian musical terms used here see The Classical Music of North India, (Khan and Ruckert 341-353).
There are many kinds of raga, some rely more heavily than others on the ascending and descending patterns (Moutal). For the sake of this investigation I have chosen Rag Gour Sarang, which shows much of its melodic movement in, and relies greatly upon, its ascending and descending patterns. One version is as follows:
Ascending:
Descending:
The descending pattern is more complex and is called "vakra" or "crooked." The catch phrase (pakad) is contained in the descending pattern:
A chalan is usually a series of phrases that, in a condensed manner, typify the melodic movement of a particular raga, in this case Gour Sarang:
N S G R m G - R G R m G R S - N S N R S N D P - M P N S - N S G R S N - N S G R m G P M D P M P m G R G R m G R S - G m P N D P - N D P M P m G R G R m G R S - G m P N S - N S G R S N D P M P m G R G R m G R S N S
There are many types of fixed composition, for the purposes here a short instrumental gat has been chosen.
On analysis of the above melodic patterns, one begins to realize that the rules of melodic motion which they illustrate could be restated as possibilities or restrictions of movement from the perspective of each note. For example, if one starts with the note G, one sees from the above that the G may move to B, A, F#, and F. Not all notes have as much freedom of movement; F#, as seen above, can only move to G.
A convenient way to notate these rules of melodic motion is to adapt Noam Chomsky's representation of the rules of grammar in the form of rewrite rules (Holtzman 100-102; Chomsky sect. 3.2, 56-68). Following Chomsky, rewrite rules for our purpose will be written as follows:
[character -> character]
The arrow is interpreted as an instruction to replace the character to the left of the arrow with a character to the right of the arrow. Using the above examples of melodic motion in raga Gour Sarang, the possible movement from the note G would be shown as:
[G -> F, F#, A, B]
This form of notation is not only convenient and compact but also represents a very helpful way to think about the melodic movement of a raga when trying to program a computer to improvise within the restrictions of such movement.
The set of rewrite rules used for the first program are as follows:
| Indian Sargam | Western note names |
| [S -> N, S, R, G] | [C -> B, C, D, E] |
| [R -> S, R, G] | [D -> C, D, E] |
| [G -> R, G, m] | [E -> D, E, F] |
| [m -> G, m, P | ][F -> E, F, G] |
| [M -> P] | [F# -> G] |
| [P -> m, M, P, D, N] | [G -> F, F#, G, A, B] |
| [D -> M, P, D] | [A -> F#, G, A] |
| [N -> D, N, S] | [B -> A, B, C] |
This simple set of rules does not fully represent the melodic possibilities of rag Gour Sarang but provides enough of a structure to test a program4.
The first program was written to simply implement these rewrite rules. The program was written in such a way that it needed a starting note (any note contained in the raga). The starting note would then be detected and sent to a section in the program such as the one below (illustration#5). The starting note then randomly triggers the next note from a group of possible notes based on the above rewrite rules. The new note would then be in the position of the starting note and trigger a new note, and so on, ad infinitum, until the program is stopped.
For a more detailed understanding of how the RagaPatch works we need to see how the MIDI pitch information passes through the program. In illustration #6 there are a series of boxes, numbered 1 through 12. Each box responds to a pitch-class (1= C, 2= C#, etc.). Each box contains the rewrite rules needed to produce an output note (as in illustration #5). For example, the incoming pitch, MIDI note 60 (middle C), will activate box #1, which will then trigger the next pitch to be played. The next pitch, say MIDI note 62 (D above middle C) and its velocity will simultaneously output a new pitch, say MIDI note 59 (B below middle C) and will activate box #12, which is now ready to output the next new pitch, and so on.
Two approaches to sensing and storing pitches were followed in different versions of RagaPatch. The first and simplest was to consider all the pitches shown in the rewrite rules as pitch-classes (as in the previous example). The second was to expand the rewrite rules to include a larger gamut of pitches. The pitches used were the 28 pitches (F3-G#5), of the main sitar playing string, sensed by the pitch-to-MIDI converter of the ESP.
The larger pitch gamut meant that the stored input was reflected more closely in the output. The pitch-class approach tended to produce melodies that jumped octaves more often than was reflective of input. Each approach produced musical and interesting melodies, and each is useful in displaying different aspects of raga performance practice. Many ragas have what is known as a bidar ang, or a branch of its realization that includes wide intervallic jumps. Most often ragas move in conjunct motion, (by conjunct motion I mean pitches that are consecutive in the arohi (ascending) or avarohi (descending) patterns, regardless of the actual compass of the interval).
The RagaPatch worked remarkably well in producing melodies that followed the rewrite rules and sounded very raga-like. Many refinements were added to this basic program that allowed creating phrases of various length, and rhythmic permutation in the Indian style.
Laya is a Sanskrit musical term meaning tempo & relative speed (Khan and Ruckert). In terms of musical ability it also refers to one's ability to keep a steady tempo as well as one's ability to create complex rhythmic relationships to that tempo. Those relationships can be expressed mathematically as ratios such as: 1:1, 1.25:3, 2:3 etc. The simpler relationships are: barabar 1:1, dugun 2:1, tigun 3:1, chargun 4:1, panchgun 5:1, chegun 6:1, satgun 7:1, atgun 8:1, naugun 9:1, dasgun 10:1.
The concepts of laya, utilizing the simpler relationships mentioned above, have been implemented as part of RagaPatch in the LayaPatch shown below. For use in programs created in MAX, basic tempo is expressed in milliseconds. The basic tempo can be set or changed at any time; the relative speeds can be set and changed specifically or randomly within a specified range; also, by using the counter object, phrases of a specific number of notes can be set. All parameters of the LayaPatch are accessible and can be changed by external input.
Next I added a way for the RagaPatch to detect when a phrase has begun and ended. The beginning of a phrase is marked by the detection of pitch information. The end of a phrase is indicated by a pause of a minimum length, a pause being the time between the last note-off and the next note-on. The simple solution was to time pauses in a section of the program I call the Phrase Follower, a portion of which is shown in illustration #8. With experimentation I found that a useable minimum time for such a pause to indicate the end of a phrase was 750 milliseconds (three-quarters of a second). This provided means of trading phrases with RagaPatch.
Meend is one kind of ornament in Indian classical music. It refers to the intricate bending of notes which is raga specific. Each raga will have its own kind of ornaments that will be associated with specific notes and phrases even though the same ornament may be found in different ragas. One of the methods developed to perform these note bends was to capture a performance of a meend in a coll object (define object), save it and trigger it when needed.
In order to have the program follow the rhythm of the input, I utilized two objects made by Chris Muir, TapTempoMS and WeightAvg.
A "clear" function, accessible by a foot switch or an on-screen mouse click, allows the performer to clear the memory of RagaPatch and start fresh. This simple control helps the performer shape the output of the program.
After incorporating the above and several other parts into the RagaPatch, I revised the rewrite rule concept4 and began to search for ways to weight the choice of available pitches and add velocity choices weighted toward the present (most recent velocities played). I was able to accomplish this by using a combination of Max objects5, the Histo6 object and the table7 object (illustration #10). Using the Histo object and the table object together allowed the program to keep track of incoming pitches, their velocities, and how often they were received. Pitches and velocities that are received more often have a greater chance of being chosen for output.
The use of Histo and table also allowed the program to "learn" the necessary note connections, previously represented by rewrite rules that had to be written into the program. This ability allows the program to create its own rewrite rules from "experience" and allows for more complex rules. One can improvise a raga-like melody without knowing its "rules" and the program will abstract "rules" from the input and be able to improvise within them. This worked as well as the earlier method of using pre-programmed rewrite rules, producing the same musical results, with the added benefit of greater flexibility.
Many computer music systems have been used for pedagogical purposes (Babalan, Ebcioglu, and Laske). The RagaPatch has the potential for being an expert system that could "learn" its rules for a particular raga from a master of Indian classical music. A student could then learn more about a particular raga through experimentation by improvising with the computer monitoring the student's improvisation. The computer would then be able to indicate to the student where the improvisations did not follow the perceived rules of the raga.
This is a natural extension of the traditional learning situation. It is generally agreed that all the shorthand methods of transmitting a raga (such as arohi, avarohi and chalan mentioned above) are at best incomplete and may lead to distortion of a raga if relied upon too heavily. A student needs the long time guidance of a master musician in order to learn all the intricacies of a raga. It is quite conceivable that an expert system could be used as a part of this process.
Of course there are many steps from this simple beginning to the creation of such an expert system.
The implications of abstracting these kind of rules from human performance has application outside of Indian classical music. In one experiment I used non-pitched percussion sounds and played percussion patterns into RagaPatch (via a MIDI keyboard instrument). Its improvised response was very coherent and musical.
After experimenting with several non-raga musical situations it has become clear that this concept of raga-structure is a more universal musical phenomenon. It is widely agreed that music involves syntax or the proper order of musical material. The rewrite rules used here could be applied to any aspect of sound material such as timbre, pitch, velocity, or, on a larger musical scale, the ordering of sections of a piece of music, lending coherence to computer generated music.
![[opening page]](jpegs&gifs/opening.jpeg)