Send Google Analytics Data to Your Own Server

In last week’s post, we explored how to tag individual users and hits with unique identifiers in Google Analytics, so that an analyst could export raw data from the Google Analytics API for complex statistical analyses not possible in the GA interface. But there are undoubtedly some situations in which even that solution isn’t good enough – Google limits the number of metrics and dimensions you can download in a single query, for example. What do you do then?

Luckily, there’s a solution for this. We’ll just send Google Analytics data on a little detour from the user’s browser to our own web server, process it ourselves, and query to our hearts content!

The 1945 movie, Detour, starring Tom Neal and Ann Savage.

The methodology I’ll summarize today allows an organization to leverage much of the value-add of Google Analytics (for instance, they’ve already done all the hard work of detecting JavaScript, flash, screen size, page, URL, etc.) while still processing the data on their own servers. It’s a massive win-win.

Re-directing the data

Before we get started, time to give credit where credit is due. The “secret sauce” in this post is some JavaScript code that I can’t take credit for. The code originally comes from a post by Angelfish Support on re-directing GA data to their own processing systems.

However, here’s the code snippet that will get you started – it would need to be placed between the GA setup code, and the code that actually sends the GA pageview (or other hit).

All this code does is simply hijack the “send hit” function in the GA library. When GA calls “send hit,” the code goes ahead and sends the payload over to Google, but it adds an additional request to your own server, with the same payload. You will (obviously) need to edit the URL on line 8 to match the location of your server.

Processing the data

Now, let’s generate a simple PHP script to actually grab some of that sweet, sweet Google Analytics data that we just re-directed to our server. In this case, we’ll just grab the “Document Title” parameter (encoded on the query string as “dt”). You can view the full list of Google Analytics query string parameters here.

I created a brutally simple HTML document, titled it “This is a test document” and included the GA code, plus the custom code for re-directing the output to the above PHP script. And, after a single pageview, the “output.txt” document created by PHP contained “This is a test document.” Awesome!

Of course, if you were serious about processing GA data yourself, you’d want to be inserting it into a database or something like that. I’ll leave that work up to you, though. Now that the data is flowing to your server, the sky’s the limit! I will mention, however, that doing individual database inserts every single time somebody views a page on your server might overwhelm your server very quickly – especially on a high-traffic site. If you’re concerned about this, you might consider simply serving up a static file (sans PHP) in response to the GA request, logging the full URL in Apache’s logs, and rotating your logs regularly for batch processing and bulk inserting into your database. That’ll keep the load on your server to a minimum.

Conclusion

As with anything, there are certainly ways that this could be improved and built upon. However, this is hopefully a very bare-bones illustration of how to get access to some really useful information in a much more accessible format than usual. Hope somebody puts it to good use!

9 Responses

  1. tomo March 9, 2016 / 9:52 pm

    This is definitely what i need! Thank you so much! you r awesome!

  2. rashid wahab May 9, 2016 / 3:28 pm

    Hi Dayne,

    The post was incredibly useful. Thank you. Can you advise if I need to utilise Angelfish modules to extract data, or will it be copied straight to my SQL database (assuming that is the URL).

    Kind regards
    Rashid

    • daynebatten May 9, 2016 / 3:51 pm

      Rashid,

      Definitely no need to use Angelfish modules. You’ll need to use PHP or some other server-side scripting language to parse the query string parameters coming from the HTTP request. After parsing the parameters, the code can then do a database insert to store things in your database (or, more accurately, you’ll probably have several database inserts for several different tables).

      I got half-way through building a script for this and haven’t wrapped it up. I’ll let you know if/when I get around to it!

      • rashid wahab May 10, 2016 / 6:25 am

        That’s perfect. I will see if I can create the code to parse the data and then store to the database. I will happily share with you on this site. Thanks again. The info is great.

  3. rashid wahab May 12, 2016 / 11:14 am

    Hi Dayne, could you assist me with the google code to get a copy of the _ua.gif file. I have added the extra code to the site (head tag). The website is http://www.superbirka.ru. You could review the GA code. At the moment, I am getting a _ua.gif file which is only 35bytes in size. It this correct, in your view?

    Kind regards
    Rashid

    • daynebatten May 12, 2016 / 12:42 pm

      ua.gif is just a 1-pixel gif image… It’s nothing special. I just checked and the one I receive from Google is 35 bytes, so that sounds about right.

  4. tomo June 20, 2016 / 5:25 am

    Hi Dayne´╝î can i use this way in production?

    • daynebatten June 29, 2016 / 1:09 pm

      Absolutely, as long as your server can handle the load!

Leave a Reply

Your email address will not be published. Required fields are marked *