One of the ultimate goals of music is to provide enjoyment. Be it through association of memories, stimulation, rhythm or melodic messages, music fans all over the world enjoy listening to music.
Digital media, particularly the MP3 format (and later improvements), has empowered you and me in a way imagined never before. Owning and managing a large digital song collection is much easier than owning a sizable record collection. I honestly have songs I haven’t ever heard.
The problem: listening to lots of music is harder than it seems
But actually listening to so much music is harder than managing it. Let me explain why.
There are (at least) two different strategies for listening to music. I can either:
- choose and queue some songs up in my favorite player (Rhythmbox or XMMS, at the moment)
- queue my entire music collection and turn Shuffle on in the music player.
Here’s strategy 1 in a nutshell: When I know I want to listen to Robert Miles, I queue it up and listen to it.
But, in practice, strategy 1 has problems:
- Selecting and queueing songs is a pain
- I often do not precisely know what I want to listen to
- I have to actually select a number of songs in advance, or interrupt my work every two to three minutes to queue a new song
- I do not get to discover songs I haven’t heard
I bet it happens to you too.
So, I resort to strategy 2: Shuffle play. And so, this minute Robert Miles (dream dance) is up, and the next minute perhaps UB40 (reggae) will be singing on my computer. Or Juan Luis Guerra (merengue).
You see, Shuffle is very good at helping you find previously unheard songs, but it’s rather bad at selecting the most appropriate song for your current mood. Shuffle can do wonders with a small list of songs, but small lists of songs means you do not get to experience songs which aren’t on the playlist. This means you have to either create lists (strategy 1, anyone?), or to interrupt your business every two or three minutes, maybe seven out of ten times.
I bet you have noticed this as well. Lots of music gives you wide variety, but does not guarantee enjoyment.
Unfortunately, there is no practical way to have the computer analyze all songs and tell it to play only “uptempo”, or “romantic”, or “party” type songs, other than making playlists yourself (cumbersome) or sorting by genre (useless due to the differences in song tempoes). Or is there?
How we choose what we want to listen
A party DJ keeps thousands of songs, and several playlists. But the playlists are just a small part of his equation. His brain performs, in real time, a much more complicated set of calculations: he must mix, match and weigh several factors which ultimately lead to a decision on “what to play next” (that’s the mission of the DJ). At any given minute, he factors:
- requests from people
- a good sense to determine people’s tastes in relation to music
- a good sense to determine “what to play next” based on what’s currently playing and what’s your audience’s target mood
We (you and me) perform the exact same analysis when playing music, unconsciously. Sure, we may or may not regard others’ requests, and your tastes probably differ considerably from mine, but we already silently carry around in our brains what we need:
- a list of favorite songs, divided into “mood playlists” (we decide on a whim whether a song we hear is “wanted” in our current mood)
- a huge mass of data on the relationships between songs we’ve already listened to (if asked, we can quickly know if a song would be perfect to listen to, after this one)
A mathematical foundation for the solution: network theory
Our neural network stores a song relationship network, which is intertwined with a set of playlists and preferences. Modern day players (such as Rhythmbox) already give you a way to rate songs, so preferences are mostly covered. By the way, I wish for two things in that field: Rhythmbox should build a less unobtrusive way to rate songs, and it should automatically calculate (both overall and seasonal) preferences based on what I have chosen to listen.
Song relationship networks have been explored less. There are projects which seek to mine personal networks for benefit. Audioscrobbler and AudioGalaxy have collected massive information on listeners’ preferences, and can successfully tell you that if you liked Vanessa Amorosi, you’re probably going to like Ace of Base. But they cannot find out whether it is “right” to listen to Ravine from Ace of Base after Beautiful life. Only you can.
What I’m proposing here is that Rhythmbox (or any other music player) should remember “what I’ve heard next” and use that as a reference/suggestion to choose songs in Mood mode (a Shuffle mode on steroids). All you would need to do is turn Mood on, and every once in a while hit Next until the player learns enough.
This sounds extremely difficult to implement, but it is probably easier than it seems to.
Network theory applied to music
Say you have 28 songs (A to Z) you generally like to listen to. When in a romantic mood, you like to listen to A through E, while in an uptempo mood you usually to listen to W to Z. This means A, B, C, D and E are related to each other. The same applies to W, X, Y and Z.
Relationships between songs matter just as much as personal taste. Probably Z sounds just about right after W, or X, but it may not be “wanted” after A, E or F.
It follows that the number of times you’ve heard Z after W is higher than you’ve heard it after A. Z is more strongly related to W than to A. In practical terms, the mutual relationship between W and Z goes up if you choose Z or let Z play when it comes up after W.
Songs don’t form isolated groups, though. Maybe song Q sounds great after Z and after A as well (Q has a high relationship between Z and A). Song Q forms a bridge between the two islands A-E and W-Z (this allows for smooth drifts of mood).
Mathematically speaking, the probability that Z gets queued up after W is the number of times Z played after W, divided by the the number of times songs played after W.
This system is simple. Mood mode should choose the next song to play, among a list of candidates that historically played after the current song, then assign probabilities to each candidate using a probability distribution calculated from the relationship strengths between the current and the candidate songs. If Z played 250 times after W, and X played 125 times after W, then Z has double the chance of X being queued up after W.
P(Z|W) = N(Z|W) / Sum of N(any,W)
The clash with reality: how to make this system work fine
However, this system, if implemented as explained, would not be useful (are you surprised?). You’d need to manually select songs until the computer has learned, or use Shuffle to teach the computer. For ten thousand songs, that is just not practical. It would take ages to seed the database with enough relationships for it to be useful and start “digging your tastes and moods”.
It is also not robust. What would happen if you were listening to a song for two minutes, and decide to change it to another song appropriate for the current mood? Should this event be tallied up as making the relationship stronger, or weaker?
Thus, I propose several incremental improvements for Mood mode.
Use two-way (mutual) instead of one-way (directional) relationships between songs
Whether you have heard W after Z, but not Z after W, should not matter. W and Z should still be related. That means everytime you choose Z (or let Z play) after W, the relationship should grow stronger between both Z and W.
Remember what has played in the current session
Songs that already played before should not be queued up again. This cancels out a flaw introduced by 1.
Do not increase the relationship between two songs right away
The player should expect that the user may change the song that just came up, in order to better suit his tastes. Say Z was played last, and B was chosen by the player (be it Mood mode or Shuffle mode) to play. B starts playing, and the user reacts this is not what I want to hear, let’s play X instead
. He changes to X, and lets X play to completion.
The player has to be smart enough to recognize this, perhaps by waiting until at least 25% of B has played before increasing the relationship strength between Z and B. It should then actually increase the relationship between Z and X (evidently, after 25% of X has played).
25% is off the top of my head. I don’t know the magic number. Either trial and error or statistical methods can be used to determine this.
Consider not just directly related songs, but indirectly related songs
While a friend can grant you a favor, you can also request your friend to ask another friend of his to make you one. The probability that a friend of a friend would grant you a favor is, naturally, smaller than of your friend granting a favor directly to you, but it is not zero.
M is related to N is related to P. It follows that P should generally be a better a listening choice after M than a random song, although N is better.
If the list of candidates is short (say, below 3), it should be inspected one level deeper. Once a candidate has been chosen, the player should make a random choice between the candidate (with probability = 0.5) or any of its related songs (with sum of their probabilities = 0.5).
This will help you find relatively more likable alternatives to listen after each song.
Applying this principle one level deeper may be marginally more useful, but would be much more computationally intensive. Or it may not be useful at all.
Constrain the number of candidates
Over time, the list of candidates for any song may grow very big. Constraining the list of candidates to, say, the top 5, will help by penalizing weak relationships.
Weaken relationships with time
New songs are at a disadvantage in Mood mode. Allowing the relationship network to adapt by weakening old relationships gradually would help new songs find their way into candidate lists, and would help the system in adapting to your changing tastes.
This weakening should be controlled, so new songs can have the opportunity to appear on candidate lists, but at the same time don’t make old, repeated choices be “overwhelmed”.
Allow for discovery of new songs
This system would only let you hear songs you’ve heard before. That’s not the goal – the goal is to listen to new songs as well. The user should have a slider type control for how much “Shuffleness” he wants (between “Be totally random” to “Strictly follow my past choices”).
Ideally, the default setting for this control would be to choose songs from the list of candidates with a probability of 0.8, and choose a random song with a probability of 0.2. A “Strictly follow my past choices” setting would be ideal for hands-off hosting of a party: two clicks on a dance music track, and the rest of the party would be dance type or compatible songs. “Be totally random” would allow you to be more explorative and create new relationships.
Numbers are off the top off my head, again.
Conclusion
Exploiting network theory in music players is just one of the interesting applications of mathematics in general. I’m sure someone else can point out examples of this being used in other fields. Let’s put this to use now. We’ll innovate.
Pseudo-code for song selection
candidates = topFive(currentSong.getRelationships())
candidate = randomWeightedChoice(candidates)
if candidates.length() < 3
coinToss = randomChoiceHeadsOrTails()
if coinToss = heads
candidates = topFive(candidate.getRelationships())
candidate = randomWeightedChoice(candidates)
else if coinToss = tails
pass
playCandidate = playCandidateOrRandomSong()
if playCandidate = true
player.queue (candidate)
else
player.queue (trulyRandomSong())
About the Author:
Manuel is a 24 information systems engineer student. He's juggling work, family and college. His company's Web site can be found at
http://www.amautacorp.com/.
If you would like to see your thoughts or experiences with technology published, please consider writing an article for OSNews.
Yeah, or you can just use Smart Playlists in iTunes to do all that for you. Study your options people. It’s not hidden or anything.
Smart playlists are also in Rhythmbox. Muine, the other GNOME music player, will soon have functionality similar to smart playlists – which the primary dev refers to as groups. So, yeah, shuffle, is more refined than the author implies.
iTunes Smart Playlists are, no matter what kind of combinations you are using, hardly a match for a neural network based selection system.
I found this article very interesting, and also amusing: the discussion of a mathematical equation followed up by pseudocode implementing a coin toss… kind of ironic…!
I have never used iTunes so cannot comment on how well it achieves the “what I should listen to next” aim. However, look at it from a different approach… using this method, could your “clever” music app not gradually alter the playlist to get you out of a bad mood, or cheer you up, or wake you up, or help you calm down etc…
When I can click a “cheer me up” button on my media player and it knows exactly what I want to hear, then I will know truly innovative media players have arrived.
A-Z equals 26 songs, not 28 😉
I like the idea, but am I the only one that values an _album_ ? … where the artist has spent hours deciding in which order their songs should go together. I try to keep all mp3s together by album, and shy away from “shuffle” and even peer-to-peer downloading because they destroy that.
“using this method, could your “clever” music app not gradually alter the playlist to get you out of a bad mood, or cheer you up, or wake you up, or help you calm down etc…
When I can click a “cheer me up” button on my media player and it knows exactly what I want to hear, then I will know truly innovative media players have arrived.”
Let’s patent that, quick.
On second thought, let’s not.
I like the idea, but am I the only one that values an _album_ ?
No you’re not alone on that. Some albums like “OK Computer” (Radiohead), “The Wall” (Pink Floyd) and “( )” (Sigur Rós) must be listened on the order they’re set…no wonder that these bands don’t like iTunes, it would deface their work.
I agree with Chris to a point. Many bands produce great _albums_, but others produce great collections of songs.
I usually run a shuffled playlist of the songs that play well “out of context”, and when I am in the mood for something like “The Wall”, “Disintigration” (The Cure), or “AEnemia” (Tool) that is best listened as an _album_ I play the album in order.
Again… back to the whole Mood thing. We need AI in our media players.
The whole idea behind Mood mode is to forego smart playlists, and introduce both exploratory and mood qualities to music listening. Let the computer deduct a “personalized” playlist based on song relationships.
Let’s patent that, quick.
Consider the osnews page cached as a case for prior art
Now wheres that “sarcastic mood” music button…..
“Let the computer deduct a “personalized” playlist based on song relationships.”
Several flaws here.
#1 the assumption that a song is of a certain mood. (For some people “Fck the Police” might be a high energy pick-me0up… for others, it could be a soothing mellow tune.
#2 the assumption that a song is always of a certain mood for a certain person. (“Wish You Were Here” can be depressing to me at times, and it can also be heart warming at times.)
Conclusion: Nerds, that can’t craft a good playlist (or are too lazy too) or cannot appreciate an album or a simple shuffle, just like wasting time on retarded ideas to compensate for their lack of… something.
I’ve been thinking about this for quite a while. It has some potential. Lots of potential.
I think this can achieve both goals: smart playlist and exploration. The two aren’t exclusive.
Someone has already done some work in this, I believe called the synapse media player: http://www.synapseai.com/ , and they had a winamp plugin, but it just seemed to go short of Manuel’s suggestions, or at least it seemed this way.
Another quick suggestion, you could also mine playlists for song relationships. Load your previous playlists, and add some connections between all the songs on each one.
I’m the same way, I seem to always listen to music by album. iTunes, which I no longer use (not available as Free Software
), did a pretty good job of sorting by ARTIST, then ALBUM, then TRACK NUMBER. Winamp, which is still a nice program, couldn’t do that with the playlist editor. you’d have the #1 song from each album in the first part, then the #2s, etc. I guess Media Library would fix that… oh well.
I just posted a link to this article through the iTunes feedback form on apple.com. I hope they will implement it into iTunes 5
>”Let the computer deduct a “personalized” playlist based on
>song relationships.”
>Several flaws here.
>#1 the assumption that a song is of a certain mood. (For
>some people “Fck the Police” might be a high energy
>pick-me0up… for others, it could be a soothing mellow
>tune.
Please explain what this has to do with the maths of the problem.
>#2 the assumption that a song is always of a certain mood
>for a certain person. (“Wish You Were Here” can be
>depressing to me at times, and it can also be heart warming
> at times.)
It all depends on which song you heard before (which probably depends on your mood). The network model will find the right “next” song for you, from a set of “next” songs you’ve heard after “wish you were here”. No surprise here, if you actually read the article.
>Conclusion: Nerds, that can’t craft a good playlist (or are
>too lazy too) or cannot appreciate an album or a simple
>shuffle, just like wasting time on retarded ideas to
>compensate for their lack of… something.
Ah, now I understand. You’re a troll or this post is flamebait. Or you own like 14 MP3s.
When I first ripped all my albums and started using a PC for playing music I spent hours creating playlists. But I don’t think I’ve listened to any of them more than once or twice and I hardly ever shuffle tracks.
Albums are simply much more satisfying and enjoyable, I like to hear tracks in the context that the artists intended. Even when I’m listening to singles I like to hear multiple tracks by the same band, rather than a mix of different artists. Now I don’t even bother using a music manager, it’s just as quick to keep my main MP3 directory open and right-click on the album’s folder to play them.
Have you ever watched “High Fidelity”?
I actually enjoy selecting what album i’m gonna listen today, bet Rob (the guy in the movie) loves too. And yes, i rather listen to an entire album, just like others have said here.
But i find the idea nice anyway, keep it up.
Victor.
What your talking about is basically an The Interactive Activation and Competition Network some links for more info:
http://www.itee.uq.edu.au/~cogs2010/cmc/chapters/IAC/
The beauty of this kind of network is its error redunancy and generalisation abilities. You could also easily mix into a users personal network external influences from other sources such as Audiogalaxy (and amazons what other users like feature). You could also use other data such as bpm of the song and average frequencies.
I would love to see this feature as an extension to the normal shuffle mechanism, yes it has some downfalls as other people have mentioned but it also opens up a whole spectrum of new oppurtunities for the traditional shuffle.
“Conclusion: Nerds, that can’t craft a good playlist (or are too lazy too) or cannot appreciate an album or a simple shuffle, just like wasting time on retarded ideas to compensate for their lack of… something.”
Oh please. Just because the concept doesn’t appeal to you personally doesn’t mean it it’ not a good idea. Not everyone approaches the same problem in the same way, and your solution-of-choice isn’t necessarily the best for everyone, or even for you. Some of the best ideas of the past have been judged as “wasting time” and chances are you’re benefiting from a few of those past ideas today. But hey, reject innovation if you wish.
“And no, you are the one which seems to have a lame preference for music. (Ace of Base?) I have about 900 CDs from Jazz to R&B to Hip Hop to Rock to Funk to Reggae. And this represents a very small portion of the music that I know and love.”
Are you implying your “superior” tastes make you an expert on the topic? I’ve enjoyed Ace of Base, does that make my opinion less valid? No and no. Variety, as they say, is the spice of life.
Personally, I think this idea has potential and I’d like to see an experimental implimentation of it.
“Are you implying your “superior” tastes make you an expert on the topic? I’ve enjoyed Ace of Base, does that make my opinion less valid? No and no. Variety, as they say, is the spice of life.”
He was the one who brought up my taste and selection. I have no opinion about it whatsoever.
But, yes, njoying Ace of Base makes your opinion less valid.
First off, your initial response on this topic had some good merit and deserves to be heard, so don’t think I am arguing against you on those points.
—
“He was the one who brought up my taste and selection. I have no opinion about it whatsoever.”
The size of your selection, yes, but he never mentioned your taste. You did that first. As for the size of your collection, it’s still a valid point that, for many people, the larger your music library, the harder it is to find a song that suites your mood, particularly when you yourself aren’t sure what you wanna hear. Now, this isn’t a mission critical problem, but then few problems are.
“But, yes, njoying Ace of Base makes your opinion less valid.”
To you, perhaps. It’s not uncommon to judge harshly those who differ from you, be it in tastes or whatever. People do it all the time. It’s called bigotry. But my opinion is less valid, so don’t listen to me.
—
To everyone else, don’t worry. I won’t perpetuate this any further. My tolerance of personal attacks and ingorant remarks is relatively low, however, and once in awhile I just can’t resist.
I shuffle inside a single genre at a time.
it would make sense to have the correlation be a function of play time.
Check out irate.sf.net it’s a collaborative music player that correlates your taste with that of other people and tries to download new music that you might enjoy.
I use Soundplay for my mp3 player,it has features that utilize the the advanced attributes of the BeOS file system,but basically I just drop my selected tunes in a folder and drop it on the Soundplay window,kick back and enjoy the show,there’s no typing involved unless you want to name the folders,by category,plus if I get bored I can play a few tunes backwards or mess with them using the tons of VST special FX Soundplay supports.
For XMMS there is a plugin called IMMS which basically scores the songs you listen to depending on whether you jump to them, from them etc. It then chooses among those songs with those scores as weights.
It can monitor X idleness as well so if you leave your computer while it’s playing, you won’t get wrong results.
I’ve used it since I discovered it, and it really works well once you’ve trained it, which is the only drawback, really. It can take a while training it on thousands of songs.
http://www.luminal.org/wiki/index.php/IMMS/IMMS
I really like this article…the idea is very cool and it hits home because, while I am also a lover of albums, more often than not I’m on shuffle just for the variety. And every 5 songs or so I find myself clicking “next” 20 times to find the song I really want to hear or that goes well after what I’ve just heard.
But here’s a suggestion for improvement: instead of picking an arbitrary percentage of the song, like 25%, after which you mark the relationship in an “all or nothing” kind of way, why not simply mark it proportionally to the amount of the song you’ve heard? And perhaps not even in a linear proportion. Perhaps greater than linear: for instance, if I listen to 1% of the song, the amount of relationship is, say 1 relationship unit, but if I listen to 100% of the song it is more than 100 times more, say 1000 relationship units.
Just an idea.
If I leave my PC for an hour, I don’t want this thing decided that I now like all the songs it has strung together. I think it needs to be aware if there is any activity on the PC when deciding.
Also, I like the point about “albums” and how the artist has spent time making the album order mean something. And I respect it. But I also respect my own opinion about these things, and want to shuffle. I tend to listen to music where the songs are so good that they can indeed stand alone, as well as in the context of an album. And that certainly includes OK Computer, right at the top of the list.
… I’m surprised nobody mentioned them:
http://mathworld.wolfram.com/MarkovChain.html
iTunes has a leg up in the race to implement something like this. (Oh, there’s no race? Why not?) The latest version has a “Party Shuffle” which lets you see at a glance what the last X (usually five) songs that played were, what the current song playing is, and what the next X (I use 25) songs will be. It shuffles out of whatever playlist you choose, or your entire library. Since you know the songs coming next, you can delete them so that the entire queue moves up and a new randomly-selected song is added to the end, or you can drag-and-drop rearrange the queued songs.
If Apple starts to assign weight in the “shuffle” algorithm based on your choices in this mode, it could be very useful indeed. Much better, IMO, than just waiting a certain amount of time after the songs starts.
If Song A is playing and then Song F starts and I decide it clashes and change the song, my new choice might not be based on what best follows Song A, because I’ve already had a jarring experience with Song F. Also, if I’m in shuffle mode, I’ll just click Next until I find one that’s “good enough,” or until I get bored of clicking Next and give up. iTunes’ Party Shuffle mode doesn’t completely eliminate these types of situations, but it cuts out most of it.
Shuffle is never random enough for me. It’s better in RB than it was in xmms, but I still notice patterns.
I know full random is impossibly, but there are interesting sources for random seeds; far more interesting than the current second past the minute.
Well, if already this is approached from a sort-of mathematic standpoint, let’s compute a bit more.
How many songs is this system supposed to manage? It must not be too few, because that’d be overseeable by the user herself, so no real point in an AI-like scheduling mechanism.
From my own experience, my collection used to be around 6000 songs, and, since it was collected gradually (thus I had time to know most of it), it just *started* to be un-overseeable. So, let’s just raise the bar a bit – I’m not a real music addict, so I’m sure others must have greater collections. Anyways, according to Apple’s iPod ads, 10000 songs in your pocket, let’s just use this number.
So, we have 10000 songs to manage. That means almost a fifty million individual song pairs to rate and track. Since this system is supposed to represent and follow a personal taste (so community effort doesn’t play in developing the relationship database), the user would have to listen to and rate 50 000 000 song pairs. And that just means one occasion for each one – which must be a quite low-fidelity representative of the real relationships, because the user may just not bother to skip a song she usually would, or vice versa.
I guess in order for the relationship system to be as reliable as to select *one* song and then play the right sings through the whole night automatically based on this Markov-chain, the relationships must be *really* strong and reliable. Way more than a single reinforcement/denial per pair.
Also, as the author pointed out too, this would require a significant amount of computational power to operate. Maybe not that much for one-level direct evaluation – but as mentioned in the article, a two- or more level process would have to evaluate this same 50 000 000 possibilities for each song transition…
Not to mention the storage requirements. If we only consider one byte per song pair, that’s 50 megs. This is not too much on today’s harddisks; but would this 50 megs be scanned through from the HDD for each song transition, or would it be held directly in RAM? That’d already be quite much, just for this purpose. And I guess for the system to be really useful, 1 byte per song pair would be *waaay* too few. Much more likely somewhere between 10 and 50… Just multiply it up…
Has the author considered these aspects?
I find the obsession with keeping albums together an odd one… If you’ve ever been to a concert from a given artist, it’s pretty unlikely they’ll play all of an album from beginning to end. They’ll play bits of it, in a different order most likely.
Basically: if you feel like you want to listen to an entire album in one go, listen to it. If you want a random selection of songs, don’t. I don’t see what all the fuss is about.
itunes is great you can do a lot with it but it still pisses me off because its so macish grr stupid mac users