How to create LogicalType in Apache Avro

By | December 18, 2020

I am writing this post to understand how to create a custom LogicalType in Apache Avro. It helps in enforcing strict schema validations for the data in transit as well as rest within their systems. Dividing this into broader sections:

  1. Define a schema using Avro IDL
  2. Generate Java classes using gradle avro plugin
  3. Define Custom Logical Type
  4. Test the Custom LogicalType
  5. Next Steps (There is always a better way)
LogicalType in Avro
Define schema using Avro IDL

Lets go over in detail how to create a Custom LogicalType in Apache Avro.

  1. Define a schema using Avro IDL

    Any simple to complex schema can be defined easily using Avro IDL. Maven and Gradle plugins which aid in generation .avsc files and java classes for the defined schema. Below is a sample schema defined by me. Read more about Avro IDL here. As a standard practice we usually define default value for our fields to accommodate for backward compatibility whenever schema changes.

  2. Generate Java classes and schema file

    Gradle avro plugin to our rescue. The plugin has been updated many times. You might notice that what I have is an older version. The first task creates Avro protocol files, next task created schema files (.avsc files), The final task created the required java classes.
    The complete build file can be found here. Gradle Avro Plugin

  3. Define Custom LogicalType

    A logical type is an Avro primitive or complex type with extra attributes to represent a derived type. This basically is a custom defined type and can be leveraged for many use cases. Notice the “reversed” keyword in the .avdl file in Step 1.
    We need to define the LogicalType and the custom conversion for it. For example, What I am doing here is appending the data ingested as part of ‘queryAuthor‘ field with the word ‘reversed‘.
    Use Cases:
    -Create a type for encryption which encrypts the data ingested as well as decrypts the field later.
    -Create definition for Address (Line1, Line2, Line3, State etc ) and use that as Custom Logical type.
    -Choose to do a one way hash of the data ingested.
    You can find the LogicalType and Conversion class here:
    Logical Type: Click here
    Conversion: Click Here

    Another important steps is to register this Logical Type which i am doing in the main method.
    public class AvroExample {
    public static void main(String[] args) {
    LogicalTypes.register(ReversedLogicalType.REVERSED_LOGICAL_TYPE_NAME, new LogicalTypes.LogicalTypeFactory() {
    private final LogicalType reversedLogicalType = new ReversedLogicalType();
    public LogicalType fromSchema(Schema schema) {
    return reversedLogicalType;

    What I am doing in the Custom conversion is self explanatory if you look at the code below.
    Image of Custom conversion class

  4. Test the Custom LogicalType

    My approach to test the Logical Type is:
    a) Write data to .avro file.
    b) Read the data in the file.
    c) Read the files using avro tools

    Better explained with some screenshots:
    In the main method I am creating an avro file and we shall see the value before the data is saved as .avro file. Then we read the data from the file and print it before conversion happens.

    Testing Logical Type
    We also see the data in the avro file using avro-tools.

  5. Next Steps or TODO:

    There is always a better way to do it. I am going to work on these and probably come up with another post
    – Github link is here.
    – Leverage latest version of plugin to register the LogicalType as part of the build.
    – Notice the TODO in
    – Explore modifying velocity templates to include the LogicalTypes in the auto-generated code.

    – Some of my other learnings.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.