AnsweredAssumed Answered

Best methods for importing/storing/reporting JSON time series data

Question asked by thefedorcompany on Jan 18, 2017
Latest reply on Jan 19, 2017 by jbante

Hi there, Everyone!

 

Looking for some help/suggestions regarding some best practices and recommendations for dealing with time series sensor data.

 

I'm looking to take JSON data from an API, store it, in some form, in Filemaker, and then visualize it.

 

The data itself is essentially just large collections of name/value pairs (a timestamp and a sensor value). The problem is dealing with the volume of data. I'm pulling values every 30 seconds from 4 tags from 3 devices 24/7 which equates to just about 35,000 data points a day.

 

I don't think storing each name/value pair in its own record is a good option; that's just a crazy amount of data to be doing finds on to bring into charts for visualization.

 

Looking for alternatives to just storing every data point as its own record and maybe even alternatives to visualizing/charting the data outside of Filemaker's options (for example, I've used Google Charts previously to do some visualization in a web viewer). Specifically looking for alternatives that make the data quickly findable and retrievable for display.

 

I attached a copy of some of the type of JSON data that I'm working with that's returned to me from the API calls.

 

Be really interested to hear any ideas/successful experiences/experiments that you all have done.

 

Thanks tons in advance!

Outcomes