How to pivot large tables in BigQuery?

Pivoting a table is a very common operation in data processing. But there is no direct function in BigQuery to perform such operation. To solve this problem I have written a Python module, BqPivot. It generates a SQL query to pivot a table that can then be run in BigQuery. In this blog post, I will … Continue reading How to pivot large tables in BigQuery?

6 Things Every Beginner Should Know To Write Clean Python Code

I have studied Java at my high school. When I first started writing Python in my freshman year, I used to mentally translate Java to Python. But after some good amount of open source exposure, I figured that Python is way cleaner and idiomatic than Java. In this blog post, I discuss 6 things I wish … Continue reading 6 Things Every Beginner Should Know To Write Clean Python Code

Hotdog or Not Hotdog – Image Classification in Python using fastai

Earlier, I was of the opinion that getting computers to recognize images requires - huge amount of data, carefully experimented neural network architectures and lots of coding. But, after taking the deep learning course - fast.ai, I found out that it is not always true. We can achieve a lot by writing just a few lines … Continue reading Hotdog or Not Hotdog – Image Classification in Python using fastai

How to One Hot Encode Categorical Variables of a Large Dataset in Python?

In this post, I will discuss a very common problem that we face when dealing with a machine learning task - How to handle categorical data especially when the entire dataset is too large to fit in memory? I will talk about how to represent categorical variables, the common problems we face while one hot … Continue reading How to One Hot Encode Categorical Variables of a Large Dataset in Python?