Thursday, June 23, 2016

How is Data Security managed in StatsPanda.com?

It goes without saying for any application worth it's salt, Data Security is THE most critical aspect of overall engineering and StatsPanda.com is no exception. StatsPanda achieves this by implementing multi tenancy support. Under the hood it uses Multi-Tenancy Technology available on Google Cloud Computing Infrastructure. At a high level data is organized under dedicated Namespaces owned by the respective Organizations and are classified under four types of visibility. 

  • public - anyone can view, listed.
  • limited - anyone can view, not listed.
  • protected - anyone from owner Organization can view
  • private - only creator can view

This premise is applicable for all the Charts, Dashboards or Datasources created in StatsPanda.com. 

API access is always restricted to a logged in user. A logged in user can create or view data in it's Org's namespace as well as in default namespace.  


Viewing Charts and Dashboard are more relaxed. Anyone can view the Charts and Dashboards in default namespace as in the ones with 'limited' visibility.


Detailed Data Security implementation philosophy is explained below.


Public : An end user can view this without logging into StatsPanda.com. A logged in end user can set visibility to 'public' and it will be created under default namespace. However, only Super Admin may approve for public listing. Once approved these will appear under the tab "Public" under respective type of Charts.  

  • visibility is set to 'public'
  • Stored under respective default namespace
  • Will be listed publicly only when approved by Super Admin.

Limited : An end user can view this as long as they have the url, they don't need to login to view the charts. However, these are not listed under the "Public" tab, these are listed under tab with "Organization ID" as name.
  • visibility is set to 'public'
  • Anyone with the url can view the content
  • User need not be logged in in order to view the content
  • Admin will not "copy" charts under this category for public listing on StatsPanda.com.
  • Data is stored under respective Organization specific Namespace.

Protected: End user must be logged in and can view only the records belonging to his/her organization. These are listed under tab with "Organization ID" as name.
  • visibility is set to 'protected'
  • User must be logged into StatsPanda.com in order to view 'protected' content
  • S/he must be part of the Organization
  • Data is stored under respective Organization specific Namespace.

Private: End user must be logged in and can view only the records that were created by him/her. These Charts are listed under the tab with "User ID" as name.
  • visibility is set to 'private'
  • User must be logged into StatsPanda.com in order to view 'private' content
  • User can view only the Charts or Dashboards that are created by him/herself. 
  • Data is stored under respective Organization specific Namespace.
  • Content can not be viewed by others in Organization.

URL Patterns:

Charts & Dashboards:

Publicly Listed, stored in default space:

/ui/charts/morris/bar-chart?unique_key=<ID>

/ui/dashboard/acme-corp-financial-analysis?unique_key=<ID>

Limited/Protected/Private:

/ui/charts/<namespace>/morris/bar-chart?unique_key=<ID>

/ui/dashboard/<namespace>/acme-corp-financial-analysis?unique_key=<ID>


APIs:

Publicly Listed, stored in default space:

/apiinput/charts/morris/line-chart/jsoninput/data?unique_key=<ID>&authToken=<token>

Limited/Protected/Private:

/apiinput/charts/<namespace>/morris/line-chart/jsoninput/data?unique_key=<ID>&authToken=<token>

You may like to refer to our previous blog on RESTful APIs in StatsPanda.com for an exhaustive list of APIs.

Wednesday, May 11, 2016

Quick Introduction to RESTful APIs in StatsPanda.com

StatsPanda.com is largely RESTful API based Data Visualisation Application. Every service on StatsPanda can be used using REST API invocation leading to creation of Charts, Graphs & Dashboard. In fact, by design, the web application has not provided user interfaces for many of the services like Creating Charts, Creating Dashboards, Importing data etc.

There are different types of APIs 
  • API to get info on the API itself 
  • API to create a Chart or Graph & other core functions  
  • API to get API JSON Input or Data File info
  • API to update a Chart or Graph 
  • API to delete a Chart or Graph 
For an end users, here are the set APIs by various activities -

1. Upload a data file and then use set of READ ( list & details) and DELETE APIs

2. Data Connector APIs import data from uploaded files or other data storage sources into StatsPanda to create Data Sources. CREATE, READ (list and details) & DELETE APIs are available.

3. Use the Datasource created in step#2 or static data to create Charts. CREATE, READ (list and details), UPDATE & DELETE operations supported.

4. Finally use the Dashboard APIs to stitch the charts created in step#3 and create a dashboard. CREATE, READ (list and details), UPDATE & DELETE operations supported.

For actual file upload, use the Web Interface. Rest of the activities can be completed using the APIs listed below.


// #######################   File Uploads   ######################//

|---------------------------------------------------------------|
|  File Uploads APIs
|---------------------------------------------------------------|

CREATE

No create API. Actual file upload happens using the web interface.



|---------------------------------------------------------------|
|  Fetches the list of Uploaded Files
|---------------------------------------------------------------|

GET:    /apiinput/fileuploads
GET:    /apiinput/fileuploads/organization/:orgId
GET:    /apiinput/fileuploads/organization/:orgId/user/:userName

|---------------------------------------------------------------|
|  Gets the details of API Inputs as persisted
|---------------------------------------------------------------|


GET:    /apiinput/fileuploads/:uniqueKey
GET:    /apiinput/fileuploads/organization/:orgId/:uniqueKey
GET:    /apiinput/fileuploads/organization/:orgId/user/:userName/:uniqueKey


API Input:

GET:    /apiinput/fileuploads/json/:uniqueKey/action/getapiinput
GET:    /apiinput/fileuploads/json/organization/:orgId/:uniqueKey/action/getapiinput
GET:    /apiinput/fileuploads/json/organization/:orgId/user/:userName/:uniqueKey/action/getapiinput

UPDATE

No Update APIs


DELETE
/apiinput/fileuploads/:uniqueKey
/apiinput/fileuploads/organization/:orgId/:uniqueKey


// ###############################################################//


// ########################   Datasource   #######################//

|---------------------------------------------------------------|
|  Get the details about the API i.e. API documentation
|---------------------------------------------------------------|

GET: /datasource/connector/json
GET: /datasource/connector/csv
GET: /datasource/connector/excel
GET: /datasource/connector/google/sheet

|---------------------------------------------------------------|
|  Create the Datasource
|---------------------------------------------------------------|

POST: /datasource/connector/json
POST: /datasource/connector/csv
POST: /datasource/connector/excel
POST: /datasource/connector/google/sheet

|---------------------------------------------------------------|
|  Url for viewing the Datasource
|---------------------------------------------------------------|

UI: /ui/charts/datasource/data-table?unique_key=:uniqueKey
UI: /ui/charts/datasource/data-table?organizationId=:orgId&unique_key=:uniqueKey


NO PUT


|---------------------------------------------------------------|
|  Fetches the list of charts
|---------------------------------------------------------------|

GET /apiinput/datasource/json


GET /apiinput/datasource/excel/organization/:orgId

GET /apiinput/datasource/sheet/organization/:orgId/user/:userName


|---------------------------------------------------------------|    


|  Gets the details of API Inputs as persisted, used for Edit
|---------------------------------------------------------------|

GET /apiinput/datasource/json/:uniqueKey (X)


GET /apiinput/datasource/excel/organization/:orgId/:uniqueKey


GET /apiinput/datasource/sheet/organization/:orgId/user/:userName/:uniqueKey

|---------------------------------------------------------------|


| READs only the Data
|---------------------------------------------------------------|

GET /apiinput/datasource/json/:uniqueKey/action/getdata
GET /apiinput/datasource/excel/organization/:orgId/:uniqueKey/action/getdata
GET /apiinput/datasource/sheet/organization/:orgId/user/:userName/:uniqueKey/action/getdata


|---------------------------------------------------------------|


| READs the Datasource API Input: 
|---------------------------------------------------------------|


GET: /apiinput/datasource/json/:uniqueKey/action/getapiinput
GET: /apiinput/datasource/excel/organization/:orgId/:uniqueKey/action/getapiinput
GET: /apiinput/datasource/sheet/organization/:orgId/user/:userName/:uniqueKey/action/getapiinput


|---------------------------------------------------------------|




| DELETEs the Datasource API Input: 
|---------------------------------------------------------------|

DELETE: /apiinput/datasource/json/:uniqueKey
DELETE: /apiinput/datasource/excel/organization/:orgId/:uniqueKey



// ###############################################################//


// ########################## CHARTS  ###########################//


|---------------------------------------------------------------|
|  Get the details about the API i.e. API documentation
|---------------------------------------------------------------|


GET: /charts/morris/bar-chart


|---------------------------------------------------------------|
|  Create the Chart
|---------------------------------------------------------------|

POST: /charts/morris/bar-chart


|---------------------------------------------------------------|
|  Url of a Chart
|---------------------------------------------------------------|

UI: /ui/charts/morris/bar-chart?unique_key=
UI: /ui/charts/:orgId/morris/bar-chart?unique_key=


|---------------------------------------------------------------|
| Used for building the Charts, it fetches only the jsonInput Data
|---------------------------------------------------------------|

GET: /apiinput/charts/morris/bar-chart/:uniqueKey/jsoninput/data
GET: /apiinput/charts/:orgId/morris/bar-chart/:uniqueKey/jsoninput/data


|---------------------------------------------------------------|
|  Fetches the list of charts
|---------------------------------------------------------------|

GET: /apiinput/charts/bar-chart
GET: /apiinput/charts/bar-chart/:orgId
GET: /apiinput/charts/bar-chart/:orgId/:userName

|---------------------------------------------------------------|    
|  Gets the details of API Inputs as persisted, used for Edit
|---------------------------------------------------------------|

GET: /apiinput/charts/morris/bar-chart/:uniqueKey/apiinput/json
GET: /apiinput/charts/:orgId/morris/bar-chart/:uniqueKey/apiinput/json


|---------------------------------------------------------------|
|  Update the API input, used to save an updated chart
|---------------------------------------------------------------|

PUT: /apiinput/charts/morris/bar-chart/:uniqueKey/apiinput/json
PUT: /apiinput/charts/:orgId/morris/bar-chart/:uniqueKey/apiinput/json


|---------------------------------------------------------------|
|  Delete a chart
|---------------------------------------------------------------|

DELETE: /apiinput/chart/:uniqueKey
DELETE: /apiinput/:orgId/chart/:uniqueKey


// ###############################################################//


// ######################### DASHBOARD ##########################//


|---------------------------------------------------------------|
|  Get the details about the API i.e. API documentation
|---------------------------------------------------------------|

GET: /dashboard



|---------------------------------------------------------------|
|  Create the Dashboard
|---------------------------------------------------------------|

POST: /dashboard



|---------------------------------------------------------------|
|  Url of a Dashboard
|---------------------------------------------------------------|

UI: /ui/dashboard/new-public-dashboard?unique_key=:uniqueKey
UI: /ui/dashboard/:orgId/new-dashboard?unique_key=:uniqueKey


|---------------------------------------------------------------|
| Used for retrieving the Dashboard jsonInput Data
|---------------------------------------------------------------|

GET: /apiinput/dashboard/:uniqueKey/jsoninput/data
GET: /apiinput/dashboard/:orgId/:uniqueKey/jsoninput/data


|---------------------------------------------------------------|
|  Fetch the list of Dashboards
|---------------------------------------------------------------|

GET: /apiinput/dashboard
GET: /apiinput/dashboard/:orgId
GET: /apiinput/dashboard/:orgId/:userName


|---------------------------------------------------------------|    
|  Get the details of API Inputs as persisted, used for Edit
|---------------------------------------------------------------|

GET: /apiinput/dashboard/:uniqueKey/apiinput/json
GET: /apiinput/dashboard/:orgId/:uniqueKey/apiinput/json


|---------------------------------------------------------------|
|  Update the API input, used to save an updated Dashboard
|---------------------------------------------------------------|


PUT: /apiinput/dashboard/morris/bar-chart/:uniqueKey/apiinput/json
PUT: /apiinput/dashboard/:orgId/morris/bar-chart/:uniqueKey/apiinput/json


|---------------------------------------------------------------|
|  Delete a Dashboard
|---------------------------------------------------------------|

DELETE: /apiinput/dashboard/:uniqueKey
DELETE: /apiinput/dashboard/chart/:uniqueKey

// ###############################################################//


// ######################### STATISTICS ##########################//

|---------------------------------------------------------------|
|  Get the details about the API i.e. API documentation
|---------------------------------------------------------------|

GET:  /stats/descriptive
GET:  /stats/regression
GET:  /stats/regression/multiregression/OLS
GET:  /stats/regression/multiregression/GLS
GET:  /stats/ranking
GET:  /stats/covariance
GET:  /stats/correlation/pearsons
GET:  /stats/correlation/spearmans
GET:  /stats/correlation/kendalls


GET:  /stats/chisquare/statistic
GET:  /stats/chisquare/p-value
GET:  /stats/chisquare/goodness-of-fit
GET:  /stats/onewayanova/f-value
GET:  /stats/mann-whitney-u/statistic
GET:  /stats/mann-whitney-u/p-value
GET:  /stats/wilcoxon-signed-rank/statistic
GET:  /stats/wilcoxon-signed-rank/p-value

|---------------------------------------------------------------|
|  Fetch the list of Statistics API Calls
|---------------------------------------------------------------|

GET:    /apiinput/stats
GET:    /apiinput/stats/organization/:orgId
GET:    /apiinput/stats/organization/:orgId/user/:userName

|---------------------------------------------------------------|    


|  Get the details of Statistics API Calls Inputs as persisted
|---------------------------------------------------------------|

GET: /apiinput/stats/:uniqueKey
GET: /apiinput/stats/organization/:orgId/:uniqueKey
GET: /apiinput/stats/organization/:orgId/user/:userName/:uniqueKey

|---------------------------------------------------------------|    


|  Get the details of API Inputs and Output Data as persisted 
|---------------------------------------------------------------|

/apiinput/stats/json/:uniqueKey/action/getapiinput|getapiout|getdata
/apiinput/stats/json/organization/:orgId/:uniqueKey/action/getapiinput|getapiout|getdata
/apiinput/stats/json/organization/:orgId/user/:userName/:uniqueKey/action/getapiinput|getapiout|getdata



|---------------------------------------------------------------|
|  Delete a Dashboard
|---------------------------------------------------------------|

DELETE:  /apiinput/fileuploads/:uniqueKey
DELETE:  /apiinput/fileuploads/organization/:orgId/:uniqueKey

// ###############################################################//

Hope you find the post on summary of API capabilities in StatsPanda.com useful.

Monday, February 29, 2016

How To Create A Dashboard Using StatsPanda.com

In this blog we will cover how you can use your data in Spreadsheet and create a Dashboard comprised of multiple Charts and Graphs using the REST API services of StatsPanda.com.

It takes approximately 4 to 5 steps before a Dashboard is ready for share. The activities ranges from data preparation, to import of Data, to create the Charts and Graphs using the imported data and finally putting all of them together in one dashboard. We will go step by step through each of the stages for creating an Example Dashboard.


For this example, we have picked up the Financial Data of Tata Consultancy Services (TCS). TCS is a large IT Services company.

1. Data Preparation


Data preparation involved collecting the Profit and Loss Statement data, calculating the key data on per employee basis and calculating the Y-O-Y data for few key performance parameters. Data has been collected for 11 years 2005 till 2015. You can view the data on Google Sheet - TCS Financial Data  or can download the data from as MS Excel from Google Storage

2. Create Datasource by importing the Data


Next, you need to import the data into StatsPanda.com system to create Datasource. You have five options here -

  1. Import the data as CSV from public web url.
  2. Import the data as Excel from public web url.
  3. Import the data from Google Sheet. Note Google Sheet needs to be published and must be public.
  4. Upload a CSV or Excel and import the uploaded file to create the Datasource.
  5. Import the data as JSON Data.

For our example here, we will use option#4.


2.1. Upload a File to StatsPanda.com


Upload the Spreadsheet using the File Upload interface that is available under Console > File  Upload.





Once uploaded it returns the response as JSON Data as shown below. Note down uniqueKey below, this ID will be used to import data from the Excel Spreadsheet.

{
    "uniqueKey": "50eecda61eac8fddda9cdc8b6a8aef5e",
    "fileName": "Financial-Data-IndianITServices.xlsx",
    "fileSize": 38641,
    "contentType": "application\/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
}


2.2. Import Data from uploaded Excel Sheet

As mentioned above an Excel Sheet can be imported either from a public url or from an uploaded file using it's uniqueKey. For this example, we imported the data using the file uploaded above. All apis for importing are available under Console > Connectors.


The API used here is -

 http://api.statspanda.com/datasource/connector/excel  This is the most flexible and powerful of all Data Import APIs for importing data. It can import specific rows, columns, starting from specific cell. For example, we have created three different Datasource from one Sheet in the uploaded excel spreadsheet.

Here is the response returned for importing 34 rows of Profit and Loss data from the first sheet on the spreadsheet starting with row number 2 & column 1.


{
    "datasourceUrl": "/apiinput/datasource/excel/2f1b41db19d6e1606aa354cb9ad8dc4a/action/getdata",
    "spreadsheetUrl": "/ui/charts/datasource/data-table?unique_key= 2f1b41db19d6e1606aa354cb9ad8dc4a"
}

Similarly, we will import 5 rows of Per Share Data starting at row number 37 & column 1.

Note down the unique_key values in above json. These values will be used while creating charts and graphs. You can view the imported Datasource here. Note that the data has been Transposed.  Similarly, we imported Key Financial Data by Employee for Tata Consultancy Services. This unique_key is the Datasource id and will be used for creation of any chart, graph etc. The second imported Datasource can be viewed here. You can find them by going the Datasource listing page under Datasource > Excel  







3. Create Charts

Next we will create multiple charts using the three different but related Datasources created in above section. 

To keep things simple, we will use only couple of types of Chart APIs.

Chart#1 - Create a line Chart with Net Sales,  EmployeeCost, OperatingProfit. The REST API used is - http://api.statspanda.com/charts/morris/line-chart. This Chart API is available under Console > Charts.

Here is the url of the generated Chart. This url will be used while creating the Dashboard.



Chart#2 - Next, lets find out how API Console is used to create a line chart with various expenses. Here is a screenshot of the API Console after the API was invoked and it returned response and showed a preview. You can view the input to the API in the upper Textarea and the response is seen in the lower Textarea.








Here is the returned JSON response from the REST API invocation as seen in the lower Textarea:

{"chartURI":"/ui/charts/morris/line-chart?unique_key=53bae695c992917122184e224faa7c9"}

Chart#3 Plot Net Sales, Operating Profit and Employee Expenses over last 11 ( 2005 till 2015).


JSON Response :


{"chartURI":"/ui/charts/nvd3/stacked-area-chart?unique_key=3c65c7e47721fe89909d96a732c344"}

Here is the Chart when embedded in a blog independently.






Chart#4 Key Parameters Y-O-Y for Acme Corporation



Chart#5 A comprehensive chart with all key fields Year, NetSales, Total Expenses, OperatingProfit, Tax, Reported Net Profit,  Total Value Addition 


Chart#6 A chart with Average Revenue per Employee, Average Salary and Employee Strength


Chart#7 Acme Corp per share data


4. Dashboard Creation

It's the final step where the user will assemble all the Charts created so far into a Dashboard. Go to Console > Charts , pick http://api.statspanda.com/rest/dashboard , edit the input Jason Data.  Once created, you will find following url as part of the response. This url is the Dashboard we created in blog. Take a look.



Hopefully you find this blog useful. 

Saturday, September 5, 2015

How to Add Charts, Graphs and Visualization to a Blog Post

Over last few months, I have written few blog posts where I used pretty sophisticated Charts, Graphs and Visualization. They make the blog post lot more meaningful and readable, readers love those charts, tables & displays. In this post, I will share the service I use for those visuals and few quick steps on how to create a chart and add that to your blog.

I use the services of http://www.StatsPanda.com, it's in public beta and it's free for "Individual" users.

StatsPanda.com Home Page


Once you login using your Facebook account, it redirects to registration page and during registration it gives two options - "Individual" and "Enterprise". Choose "Individual".

Go to API Console. I will suggest that you spend some time exploring the listed Visualization APIs. You may have to spend some time trying out the APIs in order to understand the JSON Inputs structures. It's not super complicated, although it may take some time. Initially you can use the example input data provided along with API Documentation.

API Console to create Charts, Graphs etc


Once decided, go ahead and create the Chart of your choice and that would give you a unique url and iFrame Code for your chart. For this blog I created a Stacked Area Chart with the example data that API Console provided. In the above screenshot, you can see a greyed out box right below the chart. The content of that box is copied below for your reference. You can straight away use that in your blog.

<iframe src="http://www.statspanda.com/ui/charts/statspanda.com/nvd3/stacked-area-chart?unique_key=9eb5203a7532f3e1a21617c3877788fe" style="border: #E8E8E8 1px solid; height: 300px; width: 100%;"></iframe>

Here is an example how the Visualisation really appears in a blog once you include the iFrame code snippet to your blog.



The chart is dynamic and not a jpg pic, data is being served from the Website. In fact, if you want you can "Edit" the input JSON Data and have a different chart of same type.

Hopefully you find it useful and able to use the charts and graphs in your blogs.

Note: Republished from http://satyajit-paul.blogspot.in/2015/09/how-to-add-charts-graphs-and-visualization-to-a-blog.html

Sunday, January 25, 2015

What is StatsPanda.com?

Goal of StatsPanda is to break the technological barrier and enable the Data Scientists, Data Analysts and Business Users to do statistical analysis and create cool visuals i.e Charts & Graphs without knowing much about technology.

Here is a quick list of things that can be accomplished using StatsPanda.com:

Statistical Analysis
  • Descriptive Statistics
  • Regression Analysis
  • Correlation, Covariance Analysis

Inferential Statistics
  • Chisquare Testing
  • Anova, Mann Whitney Testing

Charts & Graphs
  • Simple Table Chart
  • Line Chart(s)
  • Area Chart(s)
  • Bar Chart(s)
  • Pie Chart(s)
  • Donut Chart(s)
  • Scatter Plot Chart(s) and more.

Finally, you can create your own Dashboard using the charts you have already created.


You can embed the Chart anywhere you want to e.g. in a blog post or you can use multiple such Charts to create a Dashboard. Example - TCS Financial Analysis.

Here is an example how a chart can be embedded in a blog.

Key Financial Metrics of TCS

Every functionality of StatsPanda is exposed via an API. These APIs can be executed on your data set using the user interface (API Console). The APIs can be invoked programmatically by the advanced users. Every API invocation will return you either the computed results (in case of Statistical Computation) or return you an unique url of the Chart you created.

I am sure you will find the services cool and useful.

Please note StatsPanda is still in early beta and very few people using the services for production. So, you may expect big changes in software design and work flow. Please check out this blog space for the latest updates.

You can connect to us @StatsPanda