Create your own all-in-one charting, screening, scanning tool with this open-source software.
Discovering Grafana, stockbee, and twitter dump.
Short version: Check out Grafana’s features here → Grafana Playground. It’s honestly the best open-source software I have found in the recent years that can reduce a lot of friction in your research and daily market-monitoring process and give you an ability to create anything you envision with respect to market.
Long version:
At my workplace, we have a quant team who work on the internal quant requirements and also the in-house quantitative research tool. As with any development teams, they have priorities, and it trickles down from most requested to least requested features. For the style of trading I do (relatively less turnover, more positional, optimizing for pnl per roundturn) there are a number of quantitative elements I wanted. These were off-beat, not in alignment with the broader approach within the company in trading. So, obviously these were the lowest priority items.
Most of my research and daily work is spread out across numerous python files and excel files. They also require regular maintenance, and updates. The outputs of the scans/screens I want were in raw-text format, which I have to sit and parse through. I also study any hypothesis I have on charts visually for my event driven trading, and with Python, I was only able to generate one or few charts at once, and if I wanted some other symbol, I had to edit the symbol variable to generate the chart.
All these were majorly time consuming and generally adding a layer of friction for my research and deep work. So I set out looking for a tool which can help solve this. Primarily I was looking to customise and beautify my scans/screens output, and then ended up elaborately describing my ideas of how to have my quantitative process and research work smoothed out.
So, I needed a tool which is a combination of Finviz (market visualization), Screener (screens/scans), Chartink (dashboards), Tradingview (for charting), TC2000 (ability to write your own custom indicators and plot them, ability to write custom queries and create screens/scans and sort them according to several parameters in table format), and few more tools.
The other use case was to build a market monitoring dashboard. This was spread out across several excel files and switching between them during the trading day was always difficult, and xlsx files usually take up more RAM, crash unexpectedly, and are a pain in the ass to manage. So, I needed a better way to do them. I wasn’t yet good at front-end work, had hacked together few charts using HighCharts package, but that was it.
So, I started going through all the tools available in the market to generate my vision of what I should do and was determined to do it myself because quant team can’t and won’t work on these things as these are custom to me.
It started off with me thinking I should learn javascript, and front end development, use a combination of datatables, highcharts dashboard, and so on. Even then, a lot of things I wanted to visualize using data, they looked like they’d take so much effort. I figured it would be one time effort to develop and then to maintain it would be easier.
A friend suggested Kibana. But it was based on Elastic Search and it’s another tech stack I’d need to add, and it wouldn’t work as it’s not open source or free. So, I then looked up the competitors/alternatives. Many people suggested PowerBi. It’s Microsoft and it’s free - but I was hesitant due to the platform dependent nature. Same goes for Tableau (which isn’t free btw and public version requires you to publish your data online).
That’s when reddit showed me Grafana in the recommendations. This was a godsend. As I explored the features, I realised this is IT - this is THE tool, the ONE tool which would help me do everything I want and more.
Check out Grafana’s features here → Grafana Playground
It’s open source and free to use in self-hosted mode. You can connect several data sources to it, and do any kind of data visualization.
If you can manage your database and keep it updated at your required timeframe and frequency, there’s no limit to what you can do.
You can create candlestick/bar charts with data. You can use CSV files. You can use a traditional database like Postgres or timeseries database like InfluxDB or TimescaleDB. There are more options.
Within a day of exploring, I was able to set up the data into a postgres database and setup basic charting application with timeframe and symbol-list connected.
I was also able to setup a dashboard with rss feed of news sources I wanted to follow, with an auto-refresh set to 5s interval, so that it refreshes the news feed every 5 seconds to keep the news close to real-time.
I imported the market monitor dashboards for different markets/products I monitor into one dashboard with the visualization. Earlier I was generating the daily metrics for the market monitor dashboard using python and then copy pasting it as a new row into the xlsx, manually applying the formatting, and all that. Yes, with some effort I could have made python do all that, but then, for long term purpose, this felt like the best solution.
All this in just one day though. I can only imagine what I’d be able to do with it, potentially fulfilling each one of my vision for simplifying my research process bringing it all under one roof, sort of like my own quantitative research, analysis, and visualization platform.
Given I know how to work with code, data, databases, this has been a one day work - but Grafana doesn’t need for you to know all that in order to do simple visualizations. If you have the data, you can figure it out within a day. If you have to maintain the data, you can put in the effort and learn to do it.
Seriously, check out Grafana. If you know Python, and if you also know how to do basic data maintenance and updates, keeping a database and maintaining it - you’d find this incredibly shorten and compress your daily work into one place, and reduce a lot of friction behind your research process.
Pradeep Bonde & StockBee
Few months back I discovered Pradeep and Qullamaggie and while it was nice seeing Qullamaggie’s achievements in the stock market converting a stake of $50k to more than $150M in 10 years, it’s Pradeep’s published blogposts and twitter that has helped me in my trading significantly in the last few months.
I don’t want to point out to specific things which helped as I’d be stifling your own creative journey into exploring and generating ideas through the things he’s published. But I’d highly recommend checking out Pradeep’s twitter, youtube videos, and his blog posts extensively.
Twitter dump
