I am writing this post to understand how to create a custom LogicalType in Apache Avro. It helps in enforcing strict schema validations for the data in transit as well as rest within their systems. Dividing this into broader sections: Define a schema using Avro IDL Generate Java classes using gradle avro plugin Define Custom… Read More »
Revisiting the year 2019, I also wanted to take time to document all the learnings from my last assignment. Here I am jotting down the learnings from my last assignment. While I learnt many things, one of the most important lesson to me goes in sync with a great man’s words “All I know is, Nothing”.
How to Create a UDF in Presto-1 This article is going to talk about how we will create a UDF in Presto. When i started looking for creating an UDF, all resources i found spoke about Maven build. But, the one i created is via a gradle build. This Article will cover the below: Project… Read More »
List all files in AWS S3 was one of the most recent requirement that I had to work on. Also, write them into a file and the easiest solution that I found was using the python boto. Below is the code that I used to make this work. While I worked on this, it also made… Read More »