Pandas to Blaze — Anaconda Enterprise 4.4 documentation

Anaconda Enterprise: Documentation

# Pandas to Blaze

This page maps pandas constructs to blaze constructs.

## Imports and Construction

    import numpy as np
    import pandas as pd
    from blaze import data, by, join, merge, concat

    # construct a DataFrame
    df = pd.DataFrame({
       'name': ['Alice', 'Bob', 'Joe', 'Bob'],
       'amount': [100, 200, 300, 400],
       'id': [1, 2, 3, 4],
    })

    # put the `df` DataFrame into a Blaze Data object
    df = data(df)

Computation | Pandas | Blaze
--- | --- | ---
Column Arithmetic |
    df.amount * 2
    df.amount * 2
Multiple Columns |
    df[['id', 'amount']]
    df[['id', 'amount']]
Selection |
    df[df.amount > 300]
    df[df.amount > 300]
Group By |
    df.groupby('name').amount.mean()
    df.groupby(['name', 'id']).amount.mean()
    by(df.name, amount=df.amount.mean())
    by(merge(df.name, df.id),
       amount=df.amount.mean())
Join |
    pd.merge(df, df2, on='name')
    join(df, df2, 'name')
Map |
    df.amount.map(lambda x: x + 1)
    df.amount.map(lambda x: x + 1,
                  'int64')
Relabel Columns |
    df.rename(columns={'name': 'alias',
                       'amount': 'dollars'})
    df.relabel(name='alias',
               amount='dollars')
Drop duplicates |
    df.drop_duplicates()
    df.name.drop_duplicates()
    df.distinct()
    df.name.distinct()
Reductions |
    df.amount.mean()
    df.amount.value_counts()
    df.amount.mean()
    df.amount.count_values()
Concatenate |
    pd.concat((df, df))
    concat(df, df)
Column Type Information |
    df.dtypes
    df.amount.dtype
    df.dshape
    df.amount.dshape

Blaze can simplify and make more readable some common IO tasks that one would want to do with pandas. These examples make use of the odo library. In many cases, blaze will able to handle datasets that can’t fit into main memory, which is something that can’t be easily done with pandas.

    from odo import odo

Operation | Pandas | Blaze
--- | --- | ---
Load directory of CSV files |
    df = pd.concat([pd.read_csv(filename)
                    for filename in
                    glob.glob('path/to/*.csv')])
    df = data('path/to/*.csv')
Save result to CSV file |
    df[df.amount < 0].to_csv('output.csv')
    odo(df[df.amount < 0],
        'output.csv')
Read from SQL database |
    df = pd.read_sql('select * from t', con='sqlite:///db.db')

    df = pd.read_sql('select * from t',
                     con=sa.create_engine('sqlite:///db.db'))
    df = data('sqlite://db.db::t')