apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-04-25 08:46:22 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Initial commit

Browse source
This commit is contained in:
Rohan Verma 2024-07-30 16:00:11 -07:00 • committed by GitHub
commit 55332d1ddb
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
168 changed files with 18456 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

377
apps/docs/documentation/configuration/fastapi.mdx Normal file
View file

@ -0,0 +1,377 @@
---
title: FastAPI config
"og:title": "FastAPI configuration and setup"
description: FastAPI configuration and setup
---
## Introduction
The backend uses [FastAPI](https://fastapi.tiangolo.com/). FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.8+ based on standard Python type hints
You can run the **api only** by running the `pnpm run dev` command from a terminal within the `api` directory. Alternatively, you can directly run the `run.py` file.
## API structure
The API root directory is structured as follows:
```
api
├── src
│ ├── api
│ ├── crud
│ ├── schemas
│ ├── __init__.py
│ ├── config.py
│ └── main.py
├── tests
├── .env
├── .env.example
├── harry-potter-db-seed-spells.csv
├── harry-potter-db-seed-users.csv
├── package.json
├── poetry.lock
├── pyproject.toml
├── requirements.txt
├── run.py
└── vercel.json
```
### Src directory
The `src` directory is where all the application code sits. Below briefly explains each folder/file
| Item | Description |
| --------- | ----------------------------------------------- |
| api | The API endpoints for the application |
| crud | The CRUD operations used within the application |
| schemas | The schemas used within the application |
| config.py | Main application configuration |
| main.py | Application |
### Root directory
The root directory contains the typical files one would expect to see in a Python project. The below files are worth describing:
| Item | Description |
| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| package.json | Not typical in a Python programme; used due to the nature of the monorepo. Running `pnpm run dev` at the project root will execute the `dev` script in this `package.json` |
| vercel.json | The configuration for deploying the FastAPI aspect of the application to Vercel. |
| \*.csv | Simple seed data for the database, sourced from the [Harry Potter API](https://hp-api.onrender.com/) |
## Dependencies
### Python 3.9
Because the project is being deployed on Vercel, Python version `3.9` must be used as this is the latest supported version of Python. For more information, read the [official documentation](https://vercel.com/docs/functions/runtimes/python).
If deploying to somewhere other than Vercel, check which version of Python you may use and adjust the project according to your needs.
### Supabase
The project uses [Supabase](https://supabase.com/) as a database. Since Planetscale [removed their free Hobby tier](https://planetscale.com/blog/planetscale-forever),
Supabase has seemed like a good alternative to use. You do not have to use Supabase with the backend, but the project is written from the perspective of using it.
<Tip>
If you are using Supabase,
[`supabase-py-async`](https://pypi.org/project/supabase-py-async/) is already
included as a project dependency within the `pyproject.toml`. If you are not
using Supabase, this can be removed.
</Tip>
### Poetry
[Poetry](https://python-poetry.org/) is used to manage the virtual environment and project dependencies.
A `requirements.txt` has been generated to enable the installation of Python packages via the `pip install` command.
If you do not use Poetry, you can remove the `poetry.lock` and `pyproject.toml` files.
<Tip>You will need a `requirements.txt` when deploying. Vercel also accepts a `Pipenv` file if you use Pipenv, otherwise, you'll need the `requirements.txt` for the `api` to build correctly</Tip>
## Adding your own endpoints
Given the project structure, there are three areas that you must be aware of when adding your own endpoints with new models. The main areas are:
- Schemas
- CRUD
- API
### Example: Creation of Spells endpoints
#### Step 1: Create a schema
<Steps>
<Step title="Add a schema">
Add a new schema to the `schemas` directory.
```python src/schemas/spell.py
from typing import ClassVar, Sequence
from pydantic import BaseModel
class Spell(BaseModel):
id: str
name: str
description: str
table_name: ClassVar[str] = "spells"
class SpellCreate(BaseModel):
id: str
name: str
description: str
class SpellUpdate(BaseModel):
id: str
name: str
description: str
class SpellSearchResults(BaseModel):
results: Sequence[Spell]
```
<Info>The `table_name` value needs to be included in the base `Spell` class. This **must** be the same as the name of the table created in Supabase. It is used in the CRUD operations to identify the table to work with.</Info>
| Item | Description |
|--------------------|---------------------------------------------------------------------------------------------|
| `Spell` | The base class describing the columns that will be in the Supabase table |
| SpellCreate | The class for creating a new `Spell` |
| SpellUpdate | The class for updating an existing `Spell` |
| SpellSearchResults | Describes how data will be returned in the API response. It will be a `Sequence` of `Spell` |
</Step>
<Step title="Add to __init__.py">
To easily import the new `Spell` classes throughout our application, they need to be added to the `src/schemas/__init__.py` file.
```python src/schemas/__init__.py
from .user import User, UserCreate, UserSearchResults, UserUpdate
from .spell import Spell, SpellCreate, SpellSearchResults, SpellUpdate
```
This allows us to import from `src/schemas` like so:
```python
from src.schemas import Spell, SpellCreate, SpellSearchResults, SpellUpdate
```
</Step>
</Steps>
#### Step 2: Setup CRUD operations
Specific CRUD operations can be created for each endpoint. However, generic CRUD operations in `src/crud/base.py` can also be used without modification.
<Steps>
<Step title="Create CRUD file">
```python src/crud/crud_spell.py
from fastapi import HTTPException
from supabase_py_async import AsyncClient
from src.crud.base import CRUDBase
from src.schemas import Spell, SpellCreate, SpellUpdate
class CRUDSpell(CRUDBase[Spell, SpellCreate, SpellUpdate]):
async def get(self, db: AsyncClient, *, id: str) -> Spell | None:
try:
return await super().get(db, id=id)
except Exception as e:
raise HTTPException(
status_code=404,
detail=f"{e.code}: Spell not found. {e.details}",
)
async def get_all(self, db: AsyncClient) -> list[Spell]:
try:
return await super().get_all(db)
except Exception as e:
raise HTTPException(
status_code=404,
detail=f"An error occurred while fetching spells. {e}",
)
async def search_all(
self, db: AsyncClient, *, field: str, search_value: str, max_results: int
) -> list[Spell]:
try:
return await super().search_all(
db, field=field, search_value=search_value, max_results=max_results
)
except Exception as e:
raise HTTPException(
status_code=404,
detail=f"An error occurred while searching for spells. {e}",
)
spell = CRUDSpell(Spell)
```
The `CRUDSpell` class inherits from the `CRUDBase` class located in `src/crud/base.py`. The `Spell`, `SpellCreate` and `SpellUpdate` classes are passed to the `CRUDBase` class to specify the types of data that will be used in the CRUD operations.
The operations that will be used in the API endpoints are functions of the `CRUDSpell` class.
Finally, `spell` is an instance of `CRUDSpell`. We import this instance to use in the API endpoints like so: `spell.get_all(db)` or `spell.get(db, id=id)`.
<Note>
**This part of the project is intended to be read-only**, however, the `SpellCreate` and `SpellUpdate` classes are created in the schema regardless as they are required by the `CRUDBase` model's function arguments. `CRUDBase` can be refactored to have optional parameters, or a read-only version of the `CRUDBase` class can be created. This is beyond the scope of this project.
</Note>
</Step>
<Step title="Add to __init__.py">
Similar to setting up the schemas, the `CRUDSpell` class needs to be added to the `src/crud/__init__.py` file.
```python src/crud/__init__.py
from .crud_user import user
from .crud_spell import spell
```
</Step>
</Steps>
#### Step 3: Create the API endpoints
<Steps>
<Step title="Create the endpoints">
Create the endpoints in the `src/api/api_v1/endpoints/spells.py` file.
```python src/api/api_v1/endpoints/spells.py
from typing import Literal, Optional, Union
from fastapi import APIRouter, HTTPException
from src.api.deps import SessionDep
from src.crud import spell
from src.schemas import Spell, SpellSearchResults
router = APIRouter()
@router.get("/get/", status_code=200, response_model=Spell)
async def get_spell(session: SessionDep, spell_id: str) -> Spell:
"""Returns a spell from a spell_id.
**Returns:**
- spell: spell object.
"""
return await spell.get(session, id=spell_id)
@router.get("/get-all/", status_code=200, response_model=list[Spell])
async def get_all_spells(session: SessionDep) -> list[Spell]:
"""Returns a list of all spells.
**Returns:**
- list[spell]: List of all spells.
"""
return await spell.get_all(session)
@router.get("/search/", status_code=200, response_model=SpellSearchResults)
async def search_spells(
session: SessionDep,
search_on: Literal["spells", "description"] = "spell",
keyword: Optional[Union[str, int]] = None,
max_results: Optional[int] = 10,
) -> SpellSearchResults:
"""
Search for spells based on a keyword and return the top `max_results` spells.
**Args:**
- keyword (str, optional): The keyword to search for. Defaults to None.
- max_results (int, optional): The maximum number of search results to return. Defaults to 10.
- search_on (str, optional): The field to perform the search on. Defaults to "email".
**Returns:**
- SpellSearchResults: Object containing a list of the top `max_results` spells that match the keyword.
"""
if not keyword:
results = await spell.get_all(session)
return SpellSearchResults(results=results)
results = await spell.search_all(
session, field=search_on, search_value=keyword, max_results=max_results
)
if not results:
raise HTTPException(
status_code=404, detail="No spells found matching the search criteria"
)
return SpellSearchResults(results=results)
```
Here we are calling `spell` object which we setup in step 2.
The `SessionDep` (located in `src/api/deps.py`) dependency is used to access the database.
The `response_model` parameter is used to specify the type of data that will be returned from the endpoint. This is used to generate TypeScript types for the frontend.
</Step>
<Step title="Add the endpoint to the router">
To include the new endpoints in the API, it must be added to the API router, located in `src/api/api_v1/api.py`.
```python src/api/api_v1/api.py
from fastapi import APIRouter
from src.api.api_v1.endpoints import users, spells
api_router = APIRouter()
api_router.include_router(users.router, prefix="/users", tags=["users"], responses={404: {"description": "Not found"}})
api_router.include_router(spells.router, prefix="/spells", tags=["spells"], responses={404: {"description": "Not found"}})
```
</Step>
</Steps>
#### Step 4: Create the table in Supabase
<Steps>
<Step title="Create the table">
Create a new table in your Supabase dashboard. It is **imperative** that the table name matches the `table_name` value in the `Spell` class in `src/schemas/spell.py`.
Columns should be added to match the `Spell` class. For example, we have the following `Spell` class:
```python
class Spell(BaseModel):
id: str
name: str
description: str
table_name: ClassVar[str] = "spells"
```
In this example, the table should be named `spells` (table names are case-sensitive) with the columns `id`, `name`, and `description`. \
\
For data types, `id` can be automatically assigned - in this example, it is set to a `uuid`. The `name` and `description` fields are strings, therefore their column's type is set to `text`. The `id` column should be the primary key.
<Tip>You do not need to add the `table_name` column as this is used in the FastAPI code to identify the table to work with.</Tip>
</Step>
<Step title="Seed the table with data">
The data can be seeded using the Supabase dashboard by uploading the `harry-potter-db-seed-spells.csv` file. For a more detailed explanation, please see the [official documentation](https://supabase.com/docs/guides/database/import-data#option-1-csv-import-via-supabase-dashboard)
</Step>
<Step title="Configure table security">
The table security should be configured to allow the FastAPI application to access the data. By default, Supabase will have [RLS (Row Level Security)](https://supabase.com/docs/guides/auth/row-level-security) enabled.\
\
If left un-configured, the database will return an empty array. As this is a simple read-only project, I am turning `RLS` off. However, for true CRUD operations, it should be configured to use authentication which is beyond the scope of this project.
\
To disable `RLS`, in the table settings, select `configure RLS` and then `disable RLS`.\
</Step>
<Step title="Add table connection to .env">
Once the table has been created, you must ensure that the `DB_URL` and `DB_API_KEY` parameters are populated in your `.env` file located in the root of the `API` directory.\
\
These values come from the Supabase dashboard by going to `settings` and then `API`. Copy the Project URL (`DB_URL`) and the Project API Key (`DB_API_KEY`).
<Info>
`DB_USERNAME` and `DB_PASSWORD` should also included in the `.env` as they are configured in the `src/config.py`. If you do not include these you will receive an error from Pydantic.\
\
Their inclusion is a pre-cursor to authentication, but they are not actually used in this project scaffold.
</Info>
</Step>
</Steps>
#### Step 5: Test your new endpoints
You can now test your new endpoints using the FastAPI Swagger UI or by making requests to the API.