Why we build APIs for our Python models using FastAPI

Machine learning models are usually wrapped in a REST API as part of the deployment. Using FastAPI, this process becomes a whole lot easier.

At Floryn, our main platform is coded in Ruby on Rails and our machine learning models in Python. Clear communication between the platform and the machine learning services is essential for integrating predictions in our workflows. In this blog I will explain why we used FastAPI for our REST API.

FastAPI

FastAPI was introduced near the end of 2018 by Sebastián Ramírez as an alternative to the well-known Python library Flask. Compared to Flask, FastAPI provides extra functionality out of the box, including …

• data validation using Pydantic;
• Starlette and Uvicorn for async capabilities;
• OpenAPI to provide a nice interface description for HTTP API’s.

Apart from extra functionaly, the documentation of FastAPI is also excellent.

Data validation

With Pydantic you can define the structure and types of the data you expect to receive. When the request body does not meet those requirements FastAPI will return a 422 status code (Unprocessable Entity). A simple example of such a definition is:

from pydantic import BaseModel

class Item(BaseModel):
  name: str
  price: float

Pydantic will check if the attributes name and price are present in the request body, and have the correct type. Any extra attributes will simply be ignored. This definition can then be used like this in your API:

from fastapi import FastAPI

app = FastAPI()

@app.post("/item/")
def save_item(item: Item):
  save_item_in_db(name=item.name, price=item.price)
  return {"message": f'Saved {item.name}'}

One might say that validation in APIs is nothing new, and it’s perfectly possible to return HTTP status codes in other libraries as well. That is true, but your code most likely starts to look like this to achieve the same functionality:

@app.route("/item/", methods=["POST"])
async def save_item(request):
 try:
   data = await request.json
 except json.JSONDecodeError:
   raise HTTPException(status_code=400, detail="Cannot decode JSON")

 name = data.get("name")
 if not isinstance(name, str):
   raise HTTPException(status_code=400, detail="Cannot extract name")

 price = data.get("price")
 if not isinstance(name, float):
   raise HTTPException(status_code=400, detail="Price is not a float")

 save_item_in_db(name=name, price=price)
 result = {"message": f'Saved {name}'}
 return JSONResponse(result)

So how is this convenient for APIs for machine learning models?

“Garbage in = garbage out” is a phrase you often hear about machine learning models. While this usually refers to the quality of the data the model is trained on, it is also relevant for the structure of the data during inference. Uncaught type or structure errors in the API lead to undesired results or hard to debug errors, so we would like to prevent this. In our case, Pydantic is especially useful since at Floryn we save the structure of the Postgresql database in a similar way for our Rails application in a structure.sql file.

For the dummy example it would look like this:

CREATE TABLE public.items (
    id bigint NOT NULL,
    name character varying,
    price double precision
)

It does not take much effort to transform this in a Pydantic model, which makes it very efficient to add data validation for our Python APIs. Do note, the integer 10 is fine for Pydantic because it can be coverted to the string '10' and will therefore not throw an error.

>>> Item(id=1, name=10, price=10)
Item(id=1, name='10', price=10.0)

This will give you an error since ‘Football’ cannot be converted to an integer.

>>> Item(id='Football', name='Football', price= 10)
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
----> 1 Item(id='Football', name= 'Football', price= 10)

ValidationError: 1 validation error for Item
id
  value is not a valid integer (type=type_error.integer)

If you want to ensure that all characters for name are alphabetic, you can set up your class as follows:

from pydantic import BaseModel, validator

class Item(BaseModel):
    id: int
    name: str
    price: float

    @validator("name")
    def name_alphabetic(cls, v):
        if not v.isalpha():
            raise ValueError("must be alphabetic")
        return v.title()

Which will give you the following traceback in case of an error:

In [15]: Item(id=10, name= 10, price= 10)
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
----> 1 Item(id=10, name= 10, price= 10)

ValidationError: 1 validation error for Item
name
  must be alphabetic (type=value_error)

Wrapping up

Using validating in Pydantic you prevent your API from silently failing and doing predictions that are based on garbage input. Imagine a use-case where we trained a model on items that all have a positive price. If we request a prediction for an item with a negative price, it will probably not be an accurate prediction. In this case Pydantic enable you to add a custom validator for price and notify the caller of the API that the request is invalid.