Superb Dashboard with WordPress and Github widgets

The other day in the office with a gorgeous view full of sunlight, I was passionately stubbled on our metrics. I thought about statistics about how cool they might be looking on the screen. Imagine extracting all data from different sources like Github, some trending messages from Twitter and unique visitors of your WordPress blog. Then imagine showing it in one place, like some big TV screen for everybody, motivating all of your team members. – No sooner said than done! I thought.

If you have read the article Reverse Engineering using Chrome you probably remember that usually, I use Chrome to debug websites. It helps me to spot the particular request and actually lets me pretend as I am some modern browser using command line tool. It copies all the necessary stuff that usually is stored in your browser: cookies, user agent, authorization tokens and allows you to get the response. In short, recall the panel Networks in Chrome? Just find the request that interest you, copy as cURL, then pass to https://jhawthorn.github.io/curl-to-ruby/ and if you lucky enough you’ll end up with JSON out of which you can extract more or less useful information, otherwise it’s possible that you may need to parse whole HTML page with Nokogiri library.

In this article, we are going to build a simple widget displaying page views of your blog hosted on WordPress. Fortunately, it’s doable extracting data from JSON. Keep in mind that we have no access to the public API even if it’s not true.

I googled for a moment and have found github.com/shopify/dashing which should help us to build the control panel with our own widgets. Sad that Shopify has discontinued the support of the project, although the project still alive as fork called Smashing.

Code

So, first of all, you may want to install smashing and create your first application.

Then, in the lib folder, you may want to create two classes – Request and WordPress. I made a class that inherits Request class, this way it allowed me quickly to implement several requests for different resources.

require 'net/http'
require 'uri'

module Requests
  class Request
    def initialize(url)
      @uri = URI.parse(url)
      @request = Net::HTTP::Get.new(@uri)
    end

    def response
      @response ||= Net::HTTP.start(
        @uri.hostname, @uri.port, req_options
      ) do |http|
        http.request(@request)
      end
    end

    private

    def req_options
      { use_ssl: @uri.scheme == 'https' }
    end
  end
end

What we have here, is simply response method that we are going to use in our child class.

require_relative 'request'
require 'json'

module Requests
  class Wordpress
    WEBSITE_URL = 'https://public-api.wordpress.com/rest/v1.1/sites/xxx/stats?http_envelope=1'.freeze

    def initialize
      super(WEBSITE_URL)
      chrome!
    end

    def stats
      JSON.parse(response.body)['body']['stats']
    end

    private

    def chrome!
      @request["Accept-Language"] = "en-GB,en-US;q=0.9,en;q=0.8"
      @request["Authorization"] = ...
      @request["Accept"] = "*/*"
      @request["User-Agent"] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.167 Safari/537.36"
      @request["Referer"] = "https://public-api.wordpress.com/wp-admin/rest-proxy/?v=2.0"
      @request["Authority"] = "public-api.wordpress.com"
      @request["Cookie"] = ...
    end
  end
end

Afterward, in the jobs folder, create a job named statistics.rb and put in there the following code.

SCHEDULER.every '30s' do
  stats = Requests::Wordpress.new.stats
  send_event(
    'page_views',
    current: stats['visitors_today'],
    last: stats['visitors_yesterday']
  )
end

Now what is left, is just to bring the statistics inside the widget like that and it’s ready!

Start the application using smashing start command

$ smashing start

Good enough, but how about tracking your Github repository stars?

As we already implemented request.rb, it needs just to create new child class named github.rb and as long as the response would be plain HTML body, we need to parse it using Nokogiri.

require_relative 'request'
require 'nokogiri'

module Requests
  class Github < Request
    def initialize(url)
      super(url)
      chrome!
    end

    def stars_count
      Nokogiri::HTML(response.body)
              .search(
                '.repohead-details-container
                li
                a.social-count'
              ).to_a[1].text.strip.tr(',', '.').to_f * 1000
    end

    private

    def chrome!
      @request["Accept-Language"] = "en-GB,en-US;q=0.9,en;q=0.8"
      @request["Upgrade-Insecure-Requests"] = "1"
      @request["User-Agent"] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36"
      @request["Accept"] = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
      @request["Cache-Control"] = "max-age=0"
      @request["Connection"] = "keep-alive"
    end
  end
end

Now create an instance and extract stars_count and that’s pretty much all you need.

CHEDULER.every '5s' do
  stats = Requests::Wordpress.new.stats
  send_event(
    'page_views',
    current: stats['visitors_today'],
    last: stats['visitors_yesterday']
  )

  stars = Requests::Github.new('https://github.com/sparklemotion/nokogiri').stars_count
  send_event('stars_count', current: stars)
end

That’s pretty much it. Kudo’s for reading! Chao.