Animated Densities

Tweet about this on TwitterShare on LinkedInShare on FacebookGoogle+Share on StumbleUponEmail to someone

I’ve just added (pseudo) random number generation for Gamma and Dirichlet distributions to the glasseye package (a necessary new part of the toolkit as I’ll be adding some Bayesian things shortly).

To celebrate I’ve created a wholly pointless new chart in the library: the animated density plot. I admit it adds nothing (apart from showing you how the distribution can slowly be built up by the randomly generated data) but sometimes it’s good to be pointless.

Here’s the gamma(5,1) distribution.

You can find the latest version of glasseye on github.

From Zero to D3

Tweet about this on TwitterShare on LinkedInShare on FacebookGoogle+Share on StumbleUponEmail to someone

Some time ago I was asked to run a short course to take analysts from javascript newbies to being capable of putting together interactive charts using d3. That’s a big ask, from them and from me! However it did work, or at the very least it set people off in the right direction while giving them some immediate and satisfying results.

I’m sharing the materials here as they may be of use to anyone who wants to get up and running quickly.

It’s two pronged attack. In the first session we go bottom up and run through how html, javascript, css and svgs all fit together. We then use that knowledge to build a simple scatterplot. In the second session we go top down using what we learnt from our basic example to unpick some more complex code and substitute in our own data. This gets us quite a long way.

I’m sharing the videos I made, the presentation that I talk though and the exercises that we went through. This is the material from session one. Session two to follow shortly.

Details of more courses run by Coppelia can be found here. If you are interested in onsite training please get in touch ([email protected])

The videos

Please watch the following short videos before the workshop class.

The javascript revolution

 

How it all fits together

 

Read more

Searching for Christmas: How to customise your Gephi diagram with D3

Tweet about this on TwitterShare on LinkedInShare on FacebookGoogle+Share on StumbleUponEmail to someone

It’s my first post for Coppelia and I was feeling festive, so decided to make a Christmas search visualisation.  You can use it to take a look at what people are Googling for this Christmas.

Print

In this post I will show you how I to customise your Gephi diagram using D3

There have been quite a few posts on Coppelia by Simon, James and Andy on Gephi and D3 so I thought I’d bring the techniques together and create something new!
Read more

Dendrograms in R2D3

Tweet about this on TwitterShare on LinkedInShare on FacebookGoogle+Share on StumbleUponEmail to someone

Hi, I’m Andrew and this is my first post for Coppelia! If you like the look of this feel free to visit my blog dinner with data (and see what happens when a data scientist hits the kitchen!)

I was excited by James’s last post on the new package R2D3, and I thought I would try to help further develop the package. This is a great new package, built by James Thomson (and in collaboration with myself and Simon Raper at Coppelia) that utilises D3 visualisations inside R. You can quickly create very striking visualisations with a just a few lines of code. This has recently been shared with a recent post, but since then a couple of updates have been made to increase the functionality.

In particular to the function D3Dendro, which creates dendrograms based on a hclust object in R. I had been working on a number of alternatives to the usual static dendrogram found in the package so far, so I thought I would add these in and describe them below.

I have created two new distinct functionalities:

  • Collapsible nodes
  • Radial output (rather than the more traditional ‘linear’ dendrogram)

You can clone the package from James’s github repository or run the following in R:


install.packages("devtools")
library(devtools)
install_github("jamesthomson/R2D3")
library(R2D3)

I will include the example in the original post, so you can easily compare the differences.

Original dendrogram:


hc < - hclust(dist(USArrests), "ave") JSON<-jsonHC(hc) D3Dendro(JSON, file_out="USArrests_Dendo.html")

Read more

An A to Z of extra features for the D3 force layout

Tweet about this on TwitterShare on LinkedInShare on FacebookGoogle+Share on StumbleUponEmail to someone

Since d3 can be a little inaccessible at times I thought I’d make things easier by starting with a basic skeleton force directed layout (Mike Bostock’s original example) and then giving you some blocks of code that can be plugged in to add various features that I have found useful.

The idea is that you can pick the features you want and slot in the code. In other words I’ve tried to make things sort of modular. The code I’ve taken from various places and adapted so thank you to everyone who has shared. I will try to provide the credits as far as I remember them!


Basic Skeleton

Here’s the basic skeleton based on Mike Bostock’s original example but with a bit of commentary to remind me exactly what is going on



A is for arrows

If you are dealing with a directed graph and you wish to use arrows to indicate direction your can just append this bit of code Read more

Converting an R HClust object into a D3.js Dendrogram

Tweet about this on TwitterShare on LinkedInShare on FacebookGoogle+Share on StumbleUponEmail to someone

Hi all I’m James. This is my first blog for Coppelia. Thanks to Simon for encouraging me to do this.

I’ve been doing a lot of hierarchical clustering in R and have started to find the the standard dendrogram plot fairly unreadable once you have over a couple of hundred records. I’ve recently been introduced to the D3.js gallery and I wondered if I could hack something better together. I found this dendrogram I liked and started to play. I soon realised in order to get my data into it I needed a nested json. Read more

Converting a dendrogram into a graph for a D3 force directed layout

Tweet about this on TwitterShare on LinkedInShare on FacebookGoogle+Share on StumbleUponEmail to someone

I wrote this code for a project that didn’t work out but I thought I’d share. It takes the dendrogram produced by hclust in R and converts it into json to be used in a D3 force directed graph (slicing the dendrogram near the top to create a couple of clusters). The dendrogram in R looks like this

Dendrogram of clustering on US arrests

Dendrogram of clustering on US arrests

And the end result in D3 is this Read more

Visualising cluster stability using Sankey diagrams

Tweet about this on TwitterShare on LinkedInShare on FacebookGoogle+Share on StumbleUponEmail to someone
The problem

I wanted a way of understanding how a clustering solution will change as more data points are added to the dataset on which it is built.

To explain this a bit more, let’s say you’ve built a segmentation on customers, or products, or tweets (something that is likely to increase) using one or other clustering solution, say hierarchical clustering. Sooner or later you’ll want to rebuild this segmentation to incorporate the new data and it would be nice to know how much the segmentation will change as a result.

One way of assessing this would be to take the data you have now, roll it back to a previous point in time and then add new chunks of data sequentially each time rebuilding the clustering solution and comparing it to the one before.

Seeing what’s going on

Having recorded the different clusters that result from incrementally adding data, the next problem is to understand what is going on. I thought a good option would be a Sankey diagram. I’ve tested this out on the US crime data that comes with R. I built seven different clustering solutions using hclust, each time adding five new data points to the original 20 point data set. I used the google charts Sankey layout which itself is derived from the D3 layout. Here’s the result. Read more

Finding neighbours in a D3 force directed layout

Tweet about this on TwitterShare on LinkedInShare on FacebookGoogle+Share on StumbleUponEmail to someone

I’ve adapted the code from this stackoverflow answer (from D3’s creator Mike Bostock) so that you can highlight neighbours in a force directed layout. Just doubleclick on a node to fade out any non-immediate neighbours and double click again to bring them back.

Machine Learning and Analytics based in London, UK