Altcoin Data (non-Bitcoins)

In the previous post, we've seen how we can retrieve Bitcoin data. Now, let's look at retrieving non-Bitcoin cryptocurrencies (aka altcoins).

infographic-crypto-comparison-preview

For this task, we'll need the Poloniex API.

Some background information about trading data from Poloniex:

  • Most altcoins cannot be bought directly with USD.
  • To acquire these coins, individuals often buy Bitcoins and then trade the Bitcoins for altcoins on cryptocurrency exchanges, such as Poloniex.

1. Retrieve Altcoin Data

First, we'll create a helper function called get_json_data() to download and cache JSON data from a provided URL. After which, we'll define a function called get_crypto_data() that will generate Poloniex API HTTP requests, and will subsequently call the helper function (in the first step) to save the resulting data.

from datetime import datetime
import pickle
import pandas as pd

base_polo_url = 'https://poloniex.com/public?command=returnChartData&currencyPair={}&start={}&end={}&period={}'
start_date = datetime.strptime('2015-01-01', '%Y-%m-%d') # get data from the start of 2015
end_date = datetime.now() # up until today
period = 86400 # pull daily data (86,400 seconds per day)

# Helper function to download and cache JSON data; returns a DF
def get_json_data(json_url, cache_path):
    try:        
        with open(cache_path, 'rb') as f:
            df = pickle.load(f)   
            print('Loaded {} from cache'.format(json_url))
    except (OSError, IOError) as e:
        print('Downloading {}'.format(json_url))
        df = pd.read_json(json_url)
        df.to_pickle(cache_path)
        print('Cached {} at {}'.format(json_url, cache_path))
    return df

# Retrieve cryptocurrency data from Poloniex
def get_crypto_data(poloniex_pair):
    json_url = base_polo_url.format(poloniex_pair, start_date.timestamp(), end_date.timestamp(), period)
    data_df = get_json_data(json_url, poloniex_pair)
    data_df = data_df.set_index('date')
    return data_df

In other words, get_crypto_data() will take a cryptocurrency pair string and return a dataframe containing the historical exchange rate of the two currencies.

Let's take a peek at BTC_ETH.

coinpair = 'BTC_{}'.format('ETH')
df = get_crypto_data(coinpair)
df.tail()
close high low open quoteVolume volume weightedAverage
date
2017-12-01 0.042447 0.044571 0.042202 0.043615 82865.313025 3589.518042 0.043317
2017-12-02 0.041811 0.042657 0.041600 0.042447 59068.068770 2484.743072 0.042066
2017-12-03 0.041286 0.042200 0.040400 0.041966 90700.958596 3744.967920 0.041289
2017-12-04 0.040090 0.041247 0.039802 0.041120 66678.957627 2702.009705 0.040523
2017-12-05 0.038974 0.040430 0.038500 0.040090 81925.967270 3220.563718 0.039311

2. Retrieve Multiple Altcoins

We'll download the exchange data for nine of the top cryptocurrencies, namely:
Ethereum, Litecoin, Ripple, Ethereum Classic, Stellar, Dash, Siacoin, Monero, and NEM.

The purpose is to:

  1. retrieve the exchange rate (to BTC) for each coin, and then
  2. use our BTC pricing data to convert this value to USD.
altcoins = ['ETH','LTC','XRP','ETC','STR','DASH','SC','XMR','XEM']

altcoin_data = {}
for altcoin in altcoins:
    coinpair = 'BTC_{}'.format(altcoin)
    crypto_price_df = get_crypto_data(coinpair)
    altcoin_data[altcoin] = crypto_price_df

Preview of the last 3 rows of Ethereum (ETH) price table:

altcoin_data['ETH'].tail(3)
close high low open quoteVolume volume weightedAverage
date
2017-12-03 0.041286 0.042200 0.040400 0.041966 90700.958596 3744.967920 0.041289
2017-12-04 0.040090 0.041247 0.039802 0.041120 66678.957627 2702.009705 0.040523
2017-12-05 0.038974 0.040430 0.038500 0.040090 81925.967270 3220.563718 0.039311

Preview of the last 3 rows of Litecoin (LTC) price table:

altcoin_data['LTC'].tail(3)
close high low open quoteVolume volume weightedAverage
date
2017-12-03 0.008939 0.009173 0.00856 0.009080 101786.315335 907.923576 0.008920
2017-12-04 0.008958 0.008993 0.00862 0.008974 72139.348527 632.520007 0.008768
2017-12-05 0.008593 0.008943 0.00850 0.008925 92821.306801 804.161438 0.008664

3. Retrieve Altcoin Prices in USD

Finally, we can now combine this BTC-altcoin exchange rate data with our Bitcoin pricing index (in a previous post) to directly calculate the historical USD values for each altcoin.

Load the variable btc_usd_datasets by pasting the code snippet found in this post about retrieving Bitcoin data.

This step involves the variable btc_usd_datasets which is about Bitcoins (instead of altcoins).

btc_usd_datasets.tail()
BITSTAMP COINBASE ITBIT KRAKEN
Date
2017-12-01 10287.373047 10293.184141 10364.119476 10296.380966
2017-12-02 10927.843358 10946.056300 10929.867866 10926.747343
2017-12-03 11249.499550 11317.806361 11241.321426 11274.703356
2017-12-04 11314.695288 11389.214464 11322.744359 11387.470641
2017-12-05 11652.183797 11730.844593 11665.127516 11691.906471

Calculate the average BTC price as a new column.

Again, this involves the variable btc_usd_datasets, to which we want to add a new column called avg_btc_price_usd.

Essentially, the new column avg_btc_price_usd is the average price of BITSTAMP, COINBASE, ITBIT, and KRAKEN.

btc_usd_datasets['avg_btc_price_usd'] = btc_usd_datasets.mean(axis=1)
btc_usd_datasets.tail()
BITSTAMP COINBASE ITBIT KRAKEN avg_btc_price_usd
Date
2017-12-01 10287.373047 10293.184141 10364.119476 10296.380966 10310.264407
2017-12-02 10927.843358 10946.056300 10929.867866 10926.747343 10932.628717
2017-12-03 11249.499550 11317.806361 11241.321426 11274.703356 11270.832673
2017-12-04 11314.695288 11389.214464 11322.744359 11387.470641 11353.531188
2017-12-05 11652.183797 11730.844593 11665.127516 11691.906471 11685.015594

We then create a new column in each altcoin dataframe with the USD prices for that coin.

This step involves the variables btc_usd_datasets (relating to Bitcoins) and altcoin_data (relating to altcoins, for obvious reasons).

# Calculate the USD price as a new column in each altcoin dataframe
for altcoin in altcoin_data.keys():
    a = altcoin_data[altcoin]['weightedAverage']
    b = btc_usd_datasets['avg_btc_price_usd']    
    altcoin_data[altcoin]['price_usd'] =  a * b

Next, we can reuse our merge_dfs_on_column() function from a previous post to create a combined dataframe of the USD price for each altcoin.

# Merge USD price of each altcoin into single dataframe 
combined_df = merge_dfs_on_column(dataframes=list(altcoin_data.values()), 
                                  labels=list(altcoin_data.keys()), 
                                  col='price_usd')

Finally, let's also add the Bitcoin prices as a final column to the combined dataframe.

# Add BTC price to the dataframe
combined_df['BTC'] = btc_usd_datasets['avg_btc_price_usd']

Now, we have a single dataframe that contains the daily USD prices for the ten cryptocurrencies that we're examining.

combined_df.tail()
DASH ETC ETH LTC SC STR XEM XMR XRP BTC
date
2017-12-01 772.036001 27.732962 446.614775 90.608253 0.005671 0.080214 0.227754 181.183101 0.241982 10310.264407
2017-12-02 767.372142 29.537448 459.889226 99.668622 0.006232 0.094567 0.249264 191.198229 0.245984 10932.628717
2017-12-03 769.450392 29.323889 465.363326 100.534588 0.008115 0.091406 0.273543 201.323896 0.246719 11270.832673
2017-12-04 755.868605 28.852502 460.075398 99.548102 0.010218 0.092077 0.274869 199.358244 0.244669 11353.531188
2017-12-05 753.562916 28.601880 459.345558 101.233600 0.009582 0.111942 0.287451 229.036588 0.239660 11685.015594

4. Plot the Data

Nothing beats plotting your data and looking at it.

So, let's reuse our df_scatter() function from a previous post to chart all of the cryptocurrency prices against one another.

# Chart all of the cryptocurrency prices
df_scatter(combined_df, 'Cryptocurrency Prices (USD)', seperate_y_axis=False, y_axis_label='Coin Value (USD)', scale='log')

5. Codes, In a Nutshell

from datetime import datetime
import pandas as pd
import pickle
import quandl

import plotly.offline as py
import plotly.graph_objs as go
py.init_notebook_mode(connected=True)


#**************#
# Altcoin Part #
#**************#

base_polo_url = 'https://poloniex.com/public?command=returnChartData&currencyPair={}&start={}&end={}&period={}'
start_date = datetime.strptime('2015-01-01', '%Y-%m-%d') # get data from the start of 2015
end_date = datetime.now() # up until today
period = 86400 # pull daily data (86,400 seconds per day)

# Helper function to download and cache JSON data; returns a DF
def get_json_data(json_url, cache_path):
    try:        
        with open(cache_path, 'rb') as f:
            df = pickle.load(f)   
            print('Loaded {} from cache'.format(json_url))
    except (OSError, IOError) as e:
        print('Downloading {}'.format(json_url))
        df = pd.read_json(json_url)
        df.to_pickle(cache_path)
        print('Cached {} at {}'.format(json_url, cache_path))
    return df

# Retrieve cryptocurrency data from Poloniex
def get_crypto_data(poloniex_pair):
    json_url = base_polo_url.format(poloniex_pair, start_date.timestamp(), end_date.timestamp(), period)
    data_df = get_json_data(json_url, poloniex_pair)
    data_df = data_df.set_index('date')
    return data_df

# Retrieve multiple cryptocurrencies/altcoins
altcoins = ['ETH','LTC','XRP','ETC','STR','DASH','SC','XMR','XEM']
altcoin_data = {}
for altcoin in altcoins:
    coinpair = 'BTC_{}'.format(altcoin)
    crypto_price_df = get_crypto_data(coinpair)
    altcoin_data[altcoin] = crypto_price_df

#**************#
# Bitcoin Part #
#**************#
    
# Define a helper function to download and cache datasets from Quandl (for BTC)
def get_quandl_data(quandl_id):
    cache_path = '{}.pkl'.format(quandl_id).replace('/','-')
    try:
        with open(cache_path, 'rb') as f:
            df = pickle.load(f)   
            print('Loaded {} from cache'.format(quandl_id))
    except (OSError, IOError) as e:
        print('Downloading {} from Quandl'.format(quandl_id))
        df = quandl.get(quandl_id, returns='pandas')
        df.to_pickle(cache_path)
        print('Cached {} at {}'.format(quandl_id, cache_path))
    return df

# Retrieve BTC data from several exchanges
exchanges = ['COINBASE', 'KRAKEN', 'BITSTAMP','ITBIT']
exchange_data = {}
for exchange in exchanges:
    exchange_code = 'BCHARTS/{}USD'.format(exchange)
    df = get_quandl_data(exchange_code)
    exchange_data[exchange] = df
    
# Helper function to merge a single column of each dataframe into a new combined dataframe
def merge_dfs_on_column(dataframes, labels, col):
    d = {labels[i]:dataframes[i][col] for i in range(len(dataframes))}       
    return pd.DataFrame(d)
    
# Merge the BTC prices into a single dataframe
btc_usd_datasets = merge_dfs_on_column(dataframes=list(exchange_data.values()), 
                                       labels=list(exchange_data.keys()), 
                                       col='Weighted Price')

# Calculate the average BTC price as a new column.
btc_usd_datasets['avg_btc_price_usd'] = btc_usd_datasets.mean(axis=1)


#************************#
# Altcoin + Bitcoin Part #
#************************#

# Calculate the USD price as a new column in each altcoin dataframe
for altcoin in altcoin_data.keys():
    a = altcoin_data[altcoin]['weightedAverage']
    b = btc_usd_datasets['avg_btc_price_usd']    
    altcoin_data[altcoin]['price_usd'] =  a * b
    
# Merge USD price of each altcoin into single dataframe 
combined_df = merge_dfs_on_column(dataframes=list(altcoin_data.values()), 
                                  labels=list(altcoin_data.keys()), 
                                  col='price_usd')

# Add BTC price to the dataframe
combined_df['BTC'] = btc_usd_datasets['avg_btc_price_usd']

#*************#
# Plotly Part #
#*************#

# Generate a scatter plot of the entire DataFrame
def df_scatter(df, title, seperate_y_axis=False, y_axis_label='', scale='linear', initial_hide=False):
    label_arr = list(df)
    series_arr = list(map(lambda col: df[col], label_arr))
    
    layout = go.Layout(
        title=title,
        legend=dict(orientation="h"),
        xaxis=dict(type='date'),
        yaxis=dict(
            title=y_axis_label,
            showticklabels= not seperate_y_axis,
            type=scale
        )
    )
    
    y_axis_config = dict(
        overlaying='y',
        showticklabels=False,
        type=scale )
    
    visibility = 'visible'
    if initial_hide:
        visibility = 'legendonly'
        
    # Form Trace For Each Series
    trace_arr = []
    for index, series in enumerate(series_arr):
        trace = go.Scatter(
            x=series.index, 
            y=series, 
            name=label_arr[index],
            visible=visibility
        )
        
        # Add seperate axis for the series
        if seperate_y_axis:
            trace['yaxis'] = 'y{}'.format(index + 1)
            layout['yaxis{}'.format(index + 1)] = y_axis_config    
        trace_arr.append(trace)

    fig = go.Figure(data=trace_arr, layout=layout)
    py.iplot(fig)

# Chart all of the cryptocurrency prices
df_scatter(combined_df, 'Cryptocurrency Prices (USD)', seperate_y_axis=False, y_axis_label='Coin Value (USD)', scale='log')

If you enjoyed this post and want to buy me a cup of coffee...

The thing is, I'll always accept a cup of coffee. So feel free to buy me one.

Cheers! ☕️