Analyzing the Billboard Hot 100 Archive

I recently downloaded the entire history of the Billboard Hot 100 chart, which has been tracking the most popular 100 US music tracks since 1958. So, I decided to see what I could learn!

Hot 100 Logo

Primary colors. Except not quite.

It only took me a couple of minutes to generate some interesting factoids. For instance, did you know…

  • That Rockin’ Around the Christmas Tree and Nat King Cole’s version of The Christmas Song both made it onto the charts for the first time in 1960… and both reappeared as recently as 2014?
  • Or that Imagine Dragon’s Radioactive spent more consecutive weeks on the chart than any song ever… at 85 weeks?
  • That the Beatles managed to have 14 songs on the Billboard Hot 100 at the same time in April of 1964? And that 5 of those songs managed to make the top 10 list simultaneously? Absolutely dominant.
  • How about that very few artists have managed to produce a #1 single, with no other charted songs… though the feat has been managed by Soulja Boy (with Crank That), Baauer (with Harlem Shake), and Daniel Powter (with Bad Day)? That’s the pinnacle of “one hit wonder” status right there.

Of course, a lot of these sorts of Hot 100 facts have already been calculated, so I decided to see what new information I could come up with!

Musical diversity

The first thing I wondered about was musical diversity. Are we moving towards a world with greater and greater variety on the charts, or where a small set of artists and songs tend to rule the airwaves?

Turns out, the number of artists making it onto the Hot 100 each year declined from 1960 through the mid-90’s, before coming roaring back into the 2000s. Check it out!

Artists by Year

The late 80’s and early 90’s were a bad time for artistic diversity on the music charts…

The same trend pretty much applied with songs as well…

Once again, bottoming out around 1990, then roaring back!

Once again, bottoming out in the 1990’s, then coming back!

As for what’s caused the trend, I don’t really know. The company has changed their methodology a lot over the years (for reasons such as the rise of online music sales and streaming), so it’s entirely possible that some of these results reflect methodological changes rather than real changes in musical popularity. Still, interesting stuff.

Survival rates

So, how long do songs tend to survive on the charts?

Well, once a song makes it on to the Billboard Hot 100, it has about a 50% chance of making it to 8 additional weeks. There’s a bizarre (and so far unexplained) drop around week 20, which leaves less than 10% of songs around, on average. Barely 1% of songs make it to a full year on the charts and, as we noted above, it’s Imagine Dragon’s Radioactive that drags that graph all the way out to 85 weeks.

Survival curve for Hot 100 songs

If you can explain that drop at Week 20, I’d love to know!

Of course, #1 singles don’t stay at #1 for long… Only 60% make it to a second consecutive week, while less than 14% make it to week 5. You can’t stay on top for long.

Survival curve for Billboard Hot 100 #1s

A much quicker dropoff…

Survival by decade

Interestingly, as the years have gone by, songs have started sticking around on the charts for more and more time. In the 1960’s, the average song charted for just 7 weeks, while in the 1990’s and 2000’s songs charted for nearly twice that! (I didn’t compute the average length for songs first charting in the 2010’s, because we’re still living through them.)

Hot 100 average weeks survived by decade

Songs tend to chart for quite a bit longer than they used to…

This is really strongly reflected in the list of longest-charting songs… if you pull a top 10 (or top 100) list of the songs that have survived on the chart the longest, they’re almost all from the 1990’s and 2000’s.

What’s next?

Of course, I could explore this information all day, but I do have other things going on in life… If you’re interested in seeing what you can learn, you can download the Billboard Hot 100 Archive and explore it as you see fit.

One disclaimer though: There’s a handful of weeks (like 1%) where there are missing songs. I’ve checked a lot of these on the Billboard site, and the songs are definitely missing in their web pages for those weeks. I’m not sure if that’s some sort of data problem on their end, or if some weird things happened to the charts in certain weeks??? At any rate, there’s not much I can do about it (as of this moment), but it’s definitely possible it would affect calculations of things like longest-running song. Keep this in mind when you’re analyzing the data.

Leave a comment if you come up with anything interesting…

13 Responses

  1. Nikki Lee February 7, 2016 / 1:54 am

    Certain things I’d like to tell you or ask you.

    1. Soulja Boy had several charting songs, including the Top 10 hit “Kiss Me Thru The Phone”. Maybe you missed because he went by the name Soulja Boy Tell Em?? Instead of Soulja Boy? Also, you definitely missed a few, such as Omi (who had Cheerleader this year). That’s just off the top of my head, and I’m sure there are dozens more.

    2. The phenomenon of The Beatles Top 5 take over is unrivaled throughout chart history. However, their 14 song week was recently surpassed by Justin Bieber, who had 16 as the main artist after the debut of his album Purpose.

    3. I have to assume that the first chart you made includes featured acts as their own artists? Otherwise it would be incredibly odd for the number of songs to decrease while the number of artists increased.

    4. The drop off at the 20th week is likely because of the (complete bullshit) “recurrent rule” they have set up where a song will drop out of the Hot 100 completely if it is 20 weeks old and outside of the Top 50.

    • daynebatten February 8, 2016 / 8:03 am

      1. You’re definitely right on Soulja Boy. Billboard has him as “Soulja Boy,” “Soulja Boy Tell’em,” and “Soulja Boy Tell ‘Em.” Actually, one of the maddening things about their archives was that inconsistent artist and song names/spellings made a lot of this really, really difficult. You’re also definitely right that there’s several dozen artists who have managed to have a #1 and no other charted songs… I didn’t mean that to be an exclusive list.

      2. Looks like the Biebs’ takeover happened just after I pulled all this data. Sad to see a record from the likes of the Beatles falling to Justin Bieber…

      3. Not totally following on the songs decreasing while artists increase thing, but, yes, I did include featured acts as their own artist for that purpose, since that’s how Billboard tracks them. However, I did throw them out for things like the “One Hit Wonder” list.

      4. Interesting! I had no idea!

      Also, I’ve got to be honest… After seeing that email address, I was 100% prepared for this comment to be spam. (Don’t worry, nobody else can see your email.)

  2. Shashwat March 30, 2017 / 12:07 pm

    Hi Dayne,

    I was wondering if you still have the database that you created.

    Thanks!

      • DLS225 November 13, 2017 / 11:38 pm

        Thank you so much for making your database available for download online! I was in the middle of compiling a similar list, but the Billboard Archives aren’t available right now. It’s been having issues for months and I’m getting tired of it. There’s another website that has the Hot 100 for each week but they charge a very high fee in order to access it, which I think is ridiculous. I had some difficulty opening the file because I don’t have Excel on my computer, but after a couple of crashes and a lot of swearing, I managed to save it to Word. I read a bit of your blog and I think it’s very interesting. Keep up the good work!

        • daynebatten November 14, 2017 / 7:13 am

          Thanks for your kind words! Glad you found this helpful!

  3. PapaVanTwee April 9, 2017 / 9:05 pm

    The archive doesn’t seem to work well with single quote marks, either. There are several songs that are separated into 2, sometimes 3 different “songs” because they are spelled different.

    • daynebatten April 10, 2017 / 8:01 am

      Yep, very true… Billboard seemed to be pretty inconsistent about some of those things, and I didn’t bother to do the clean-up work. If somebody wants to invest the time, I’d love to hear about it.

  4. Ernie Dome April 24, 2017 / 12:50 pm

    I heard several years ago that the Beatles held the record for “most top ten songs at one time”. I can’t remember the date, but there was supposed to be a week where they held 7 of the top ten slots. Not 1 thru 7, just seven of the top ten. Is this true? Thanks.

  5. Raihan Kibria June 25, 2017 / 7:01 pm

    Hi Dayne,
    beginner data science enthusiast here! I am using your dataset as a tutorial of sorts to learn the basics of the pandas library. I am trying to investigate the presence of any kind of pattern in the gender makeup of the hits (more as a toy problem than any kind of expectation that there is a pattern).

    I wrote up the first results so far here:
    https://rkibria.netlify.com/post/singers_gender_1/

    Maybe someone else finds this interesting!

    • daynebatten June 27, 2017 / 8:35 am

      Super interesting! Thanks for sharing!

      It actually makes total sense to me that female singers had a moment in the late 90s. A ton of female pop artists from that era come to mind…

      • Raihan Kibria June 29, 2017 / 5:58 pm

        Yes I suppose this was the period when Whitney Houston, Mariah Carey and Alanis Morissette, Madonna and various other high profile names were all active at the same time, looking at the 1996 data for example. Still curious that this only happened once over all those years, and then for only a short time. Quite an anomaly really!

Leave a Reply

Your email address will not be published. Required fields are marked *