API reference¶

Table¶

class piccolo.table.Table(_data: Dict[Column, Any] = None, _ignore_missing: bool = False, _exists_in_db: bool = False, **kwargs)¶

The class represents a database table. An instance represents a row.

The constructor can be used to assign column values.

Note

The _data, _ignore_missing, and _exists_in_db arguments are prefixed with an underscore to help prevent a clash with a column name which might be passed in via kwargs.

Parameters

_data –
There’s two ways of passing in the data for each column. Firstly, you can use kwargs:
```
Band(name="Pythonistas")
```
Secondly, you can pass in a dictionary which maps column classes to values:
```
Band({Band.name: 'Pythonistas'})
```
The advantage of this second approach is it’s more strongly typed, and linters such as flake8 or MyPy will more easily detect typos.
_ignore_missing – If False a ValueError will be raised if any column values haven’t been provided.
_exists_in_db – Used internally to track whether this row exists in the database.

add_m2m(*rows: Table, m2m: M2M, extra_column_values: Dict[Union[Column, str], Any] = {}) → M2MAddRelated¶

Save the row if it doesn’t already exist in the database, and insert an entry into the joining table.

>>> band = await Band.objects().get(Band.name == "Pythonistas")
>>> await band.add_m2m(
...     Genre(name="Punk rock"),
...     m2m=Band.genres
... )
[{'id': 1}]

Parameters

extra_column_values –

If the joining table has additional columns besides the two required foreign keys, you can specify the values for those additional columns. For example, if this is our joining table:

class GenreToBand(Table):
    band = ForeignKey(Band)
    genre = ForeignKey(Genre)
    reason = Text()

We can provide the reason value:

await band.add_m2m(
    Genre(name="Punk rock"),
    m2m=Band.genres,
    extra_column_values={
        "reason": "Their second album was very punk."
    }
)

classmethod all_columns(exclude: Sequence[Union[str, Column]] = None) → List[Column]¶

Used in conjunction with select queries. Just as we can use all_columns to retrieve all of the columns from a related table, we can also use it at the root of our query to get all of the columns for the root table. For example:

await Band.select(
    Band.all_columns(),
    Band.manager.all_columns()
)

This is mostly useful when the table has a lot of columns, and typing them out by hand would be tedious.

Parameters: exclude – You can request all columns, except these.

classmethod all_related(exclude: List[Union[str, ForeignKey]] = None) → List[Column]¶

Used in conjunction with objects queries. Just as we can use all_related on a ForeignKey, you can also use it for the table at the root of the query, which will return each related row as a nested object. For example:

concert = await Concert.objects(
    Concert.all_related()
)

>>> concert.band_1
<Band: 1>
>>> concert.band_2
<Band: 2>
>>> concert.venue
<Venue: 1>

This is mostly useful when the table has a lot of foreign keys, and typing them out by hand would be tedious. It’s equivalent to:

concert = await Concert.objects(
    Concert.venue,
    Concert.band_1,
    Concert.band_2
)

Parameters: exclude – You can request all columns, except these.

classmethod alter() → Alter¶

Used to modify existing tables and columns.

await Band.alter().rename_column(Band.popularity, 'rating')

classmethod count(column: Optional[Column] = None, distinct: Optional[Sequence[Column]] = None) → Count¶

Count the number of matching rows:

await Band.count().where(Band.popularity > 1000)

Parameters

column – If specified, just count rows where this column isn’t null.
distinct –
Counts the number of distinct values for these columns. For example, if we have a concerts table:
```
class Concert(Table):
    band = Varchar()
    start_date = Date()
```
With this data:

band

start_date

Pythonistas

2023-01-01

Pythonistas

2023-02-03

Rustaceans

2023-01-01

Without the distinct argument, we get the count of all rows:
```
>>> await Concert.count()
3
```
To get the number of unique concert dates:
```
>>> await Concert.count(distinct=[Concert.start_date])
2
```

classmethod create_index(columns: List[Union[Column, str]], method: IndexMethod = IndexMethod.btree, if_not_exists: bool = False) → CreateIndex¶

Create a table index. If multiple columns are specified, this refers to a multicolumn index, rather than multiple single column indexes.

await Band.create_index([Band.name])

classmethod create_table(if_not_exists=False, only_default_columns=False, auto_create_schema: bool = True) → Create¶

Create table, along with all columns.

await Band.create_table()

classmethod delete(force=False) → Delete¶

Delete rows from the table.

await Band.delete().where(Band.name == 'Pythonistas')

Parameters: force – Unless set to True, deletions aren’t allowed without a where clause, to prevent accidental mass deletions.

classmethod drop_index(columns: List[Union[Column, str]], if_exists: bool = True) → DropIndex¶

Drop a table index. If multiple columns are specified, this refers to a multicolumn index, rather than multiple single column indexes.

await Band.drop_index([Band.name])

classmethod exists() → Exists¶

Use it to check if a row exists, not if the table exists.

await Band.exists().where(Band.name == 'Pythonistas')

classmethod from_dict(data: Dict[str, Any]) → TableInstance¶: Used when loading fixtures. It can be overriden by subclasses in case they have specific logic / validation which needs running when loading fixtures.

get_m2m(m2m: M2M) → M2MGetRelated¶

Get all matching rows via the join table.

>>> band = await Band.objects().get(Band.name == "Pythonistas")
>>> await band.get_m2m(Band.genres)
[<Genre: 1>, <Genre: 2>]

classmethod get_readable() → Readable¶: Creates a readable representation of the row.

get_related(foreign_key: Union[ForeignKey, str]) → First[Table]¶

Used to fetch a Table instance, for the target of a foreign key.

band = await Band.objects().first()
manager = await band.get_related(Band.manager)
>>> print(manager.name)
'Guido'

It can only follow foreign keys one level currently. i.e. Band.manager, but not Band.manager.x.y.z.

classmethod indexes() → Indexes¶

Returns a list of the indexes for this tables.

await Band.indexes()

classmethod insert(*rows: TableInstance) → Insert[TableInstance]¶

Insert rows into the database.

await Band.insert(
    Band(name="Pythonistas", popularity=500, manager=1)
)

classmethod objects(*prefetch: Union[ForeignKey, List[ForeignKey]]) → Objects[TableInstance]¶

Returns a list of table instances (each representing a row), which you can modify and then call ‘save’ on, or can delete by calling ‘remove’.

pythonistas = await Band.objects().where(
    Band.name == 'Pythonistas'
).first()

pythonistas.name = 'Pythonistas Reborn'

await pythonistas.save()

# Or to remove it from the database:
await pythonistas.remove()

Parameters

prefetch –

Rather than returning the primary key value of this related table, a nested object will be returned for the row on the related table.

# Without nested
band = await Band.objects().first()
>>> band.manager
1

# With nested
band = await Band.objects(Band.manager).first()
>>> band.manager
<Band 1>

property querystring: QueryString¶: Used when inserting rows.

classmethod raw(sql: str, *args: Any) → Raw¶

Execute raw SQL queries on the underlying engine - use with caution!

await Band.raw('select * from band')

Or passing in parameters:

await Band.raw("SELECT * FROM band WHERE name = {}", 'Pythonistas')

classmethod ref(column_name: str) → Column¶

Used to get a copy of a column from a table referenced by a ForeignKey column. It’s unlikely an end user of this library will ever need to do this, but other libraries built on top of Piccolo may need this functionality.

Band.ref('manager.name')

refresh(columns: Optional[Sequence[Column]] = None) → Refresh¶

Used to fetch the latest data for this instance from the database. Modifies the instance in place, but also returns it as a convenience.

Parameters: columns – If you only want to refresh certain columns, specify them here. Otherwise all columns are refreshed.

Example usage:

# Get an instance from the database.
instance = await Band.objects.first()

# Later on we can refresh this instance with the latest data
# from the database, in case it has gotten stale.
await instance.refresh()

# Alternatively, running it synchronously:
instance.refresh().run_sync()

remove() → Delete¶: A proxy to a delete query.

remove_m2m(*rows: Table, m2m: M2M) → M2MRemoveRelated¶

Remove the rows from the joining table.

>>> band = await Band.objects().get(Band.name == "Pythonistas")
>>> genre = await Genre.objects().get(Genre.name == "Rock")
>>> await band.remove_m2m(
...     genre,
...     m2m=Band.genres
... )

save(columns: Optional[Sequence[Union[str, Column]]] = None) → Union[Insert, Update]¶

A proxy to an insert or update query.

Parameters

columns –

Only the specified columns will be synced back to the database when doing an update. For example:

band = await Band.objects().first()
band.popularity = 2000
await band.save(columns=[Band.popularity])

If columns=None (the default) then all columns will be synced back to the database.

classmethod select(*columns: t.Union[Selectable, str], exclude_secrets=False) → Select¶

Get data in the form of a list of dictionaries, with each dictionary representing a row.

These are all equivalent:

await Band.select().columns(Band.name)
await Band.select(Band.name)
await Band.select('name')

Parameters: exclude_secrets – If True, any columns with secret=True are omitted from the response. For example, we use this for the password column of BaseUser. Even though the passwords are hashed, you still don’t want them being passed over the network if avoidable.

classmethod table_exists() → TableExists¶

Check if the table exists in the database.

await Band.table_exists()

to_dict(*columns: Column) → Dict[str, Any]¶

A convenience method which returns a dictionary, mapping column names to values for this table instance.

instance = await Manager.objects().get(
    Manager.name == 'Guido'
)

>>> instance.to_dict()
{'id': 1, 'name': 'Guido'}

If the columns argument is provided, only those columns are included in the output. It also works with column aliases.

>>> instance.to_dict(Manager.id, Manager.name.as_alias('title'))
{'id': 1, 'title': 'Guido'}

classmethod update(values: Dict[Union[Column, str], Any] = None, force: bool = False, use_auto_update: bool = True, **kwargs) → Update¶

Update rows.

All of the following work, though the first is preferable:

await Band.update(
    {Band.name: "Spamalot"}
).where(
    Band.name == "Pythonistas"
)

await Band.update(
    {"name": "Spamalot"}
).where(
    Band.name == "Pythonistas"
)

await Band.update(
    name="Spamalot"
).where(
    Band.name == "Pythonistas"
)

Parameters

force – Unless set to True, updates aren’t allowed without a where clause, to prevent accidental mass overriding of data.
use_auto_update – Whether to use the auto_update values on any columns. See the auto_update argument on Column for more information.

SchemaManager¶

class piccolo.schema.SchemaManager(db: Optional[Engine] = None)¶

A useful utility class for interacting with schemas.

Parameters: db – Used to execute the database queries. If not specified, we try and import it from piccolo_conf.py.

create_schema(schema_name: str, *, if_not_exists: bool = True) → CreateSchema¶

Creates the specified schema:

>>> await SchemaManager().create_schema(schema_name="music")

Parameters

schema_name – The name of the schema to create.
if_not_exists – No error will be raised if the schema already exists.

drop_schema(schema_name: str, *, if_exists: bool = True, cascade: bool = False) → DropSchema¶

Drops the specified schema:

>>> await SchemaManager().drop_schema(schema_name="music")

Parameters

schema_name – The name of the schema to drop.
if_exists – No error will be raised if the schema doesn’t exist.
cascade – If True then it will automatically drop the tables within the schema.

list_schemas() → ListSchemas¶

Returns the name of each schema in the database:

>>> await SchemaManager().list_schemas()
['public', 'schema_1']

list_tables(schema_name: str) → ListTables¶

Returns the name of each table in the given schema:

>>> await SchemaManager().list_tables(schema_name="music")
['band', 'manager']

Parameters: schema_name – List the tables in this schema.

move_table(table_name: str, new_schema: str, current_schema: Optional[str] = None) → MoveTable¶

Moves a table to a different schema:

>>> await SchemaManager().move_schema(
...     table_name='my_table',
...     new_schema='schema_1'
... )

Parameters

table_name – The name of the table to move.
new_schema – The name of the scheam you want to move the table too.

Current_schema

If not specified, 'public' is assumed.

rename_schema(schema_name: str, new_schema_name: str) → RenameSchema¶

Rename the schema:

>>> await SchemaManager().rename_schema(
...     schema_name="music",
...     new_schema_name="music_info"
... )

Parameters

schema_name – The current name of the schema.
new_schema_name – What to rename the schema to.

Column¶

class piccolo.columns.base.Column(null: bool = False, primary_key: bool = False, unique: bool = False, index: bool = False, index_method: IndexMethod = IndexMethod.btree, required: bool = False, help_text: Optional[str] = None, choices: Optional[Type[Enum]] = None, db_column_name: Optional[str] = None, secret: bool = False, auto_update: Any = Ellipsis, **kwargs)¶

All other columns inherit from Column. Don’t use it directly.

The following arguments apply to all column types:

Parameters

null – Whether the column is nullable.
primary_key – If set, the column is used as a primary key.
default – The column value to use if not specified by the user.
unique – If set, a unique constraint will be added to the column.
index – Whether an index is created for the column, which can improve the speed of selects, but can slow down inserts.
index_method – If index is set to True, this specifies what type of index is created.
required – This isn’t used by the database - it’s to indicate to other tools that the user must provide this value. Example uses are in serialisers for API endpoints, and form fields.
help_text – This provides some context about what the column is being used for. For example, for a Decimal column called value, it could say 'The units are millions of dollars'. The database doesn’t use this value, but tools such as Piccolo Admin use it to show a tooltip in the GUI.
choices – An optional Enum - when specified, other tools such as Piccolo Admin will render the available options in the GUI.
db_column_name –
If specified, you can override the name used for the column in the database. The main reason for this is when using a legacy database, with a problematic column name (for example 'class', which is a reserved Python keyword). Here’s an example:
```
class MyTable(Table):
    class_ = Varchar(db_column_name="class")

>>> await MyTable.select(MyTable.class_)
[{'id': 1, 'class': 'test'}]
```
This is an advanced feature which you should only need in niche situations.

secret –

If secret=True is specified, it allows a user to automatically omit any fields when doing a select query, to help prevent inadvertent leakage of sensitive data.

class Band(Table):
    name = Varchar()
    net_worth = Integer(secret=True)

>>> await Band.select(exclude_secrets=True)
[{'name': 'Pythonistas'}]

auto_update –

Allows you to specify a value to set this column to each time it is updated (via MyTable.update, or MyTable.save on an existing row). A common use case is having a modified_on column.

class Band(Table):
    name = Varchar()
    popularity = Integer()
    # The value can be a function or static value:
    modified_on = Timestamp(auto_update=datetime.datetime.now)

# This will automatically set the `modified_on` column to the
# current timestamp, without having to explicitly set it:
>>> await Band.update({
...     Band.popularity: Band.popularity + 100
... }).where(Band.name == 'Pythonistas')

Note - this feature is implemented purely within the ORM. If you want similar functionality on the database level (i.e. if you plan on using raw SQL to perform updates), then you may be better off creating SQL triggers instead.

as_alias(name: str) → Column¶

Allows column names to be changed in the result of a select.

For example:

>>> await Band.select(Band.name.as_alias('title')).run()
{'title': 'Pythonistas'}

property ddl: str¶: Used when creating tables.

get_default_value() → Any¶: If the column has a default attribute, return it. If it’s callable, return the response instead.

get_select_string(engine_type: str, with_alias: bool = True) → str¶: How to refer to this column in a SQL query, taking account of any joins and aliases.

get_sql_value(value: Any) → Any¶

When using DDL statements, we can’t parameterise the values. An example is when setting the default for a column. So we have to convert from the Python type to a string representation which we can include in our DDL statements.

Parameters: value – The Python value to convert to a string usable in a DDL statement e.g. 1.
Returns: The string usable in the DDL statement e.g. ‘1’.

ilike(value: str) → Where¶

Only Postgres supports ILIKE. It’s used for case insensitive matching.

For SQLite, it’s just proxied to a LIKE query instead.

is_not_null() → Where¶: Can be used instead of MyTable.column != None, because some linters don’t like a comparison to None.

is_null() → Where¶: Can be used instead of MyTable.column == None, because some linters don’t like a comparison to None.

join_on(column: Column) → ForeignKey¶

Joins are typically performed via foreign key columns. For example, here we get the band’s name and the manager’s name:

class Manager(Table):
    name = Varchar()

class Band(Table):
    name = Varchar()
    manager = ForeignKey(Manager)

>>> await Band.select(Band.name, Band.manager.name)

The join_on method lets you join tables even when foreign keys don’t exist, by joining on a column in another table.

For example, here we want to get the manager’s email, but no foreign key exists:

class Manager(Table):
    name = Varchar(unique=True)
    email = Varchar()

class Band(Table):
    name = Varchar()
    manager_name = Varchar()

>>> await Band.select(
...     Band.name,
...     Band.manager_name.join_on(Manager.name).email
... )

like(value: str) → Where¶

Both SQLite and Postgres support LIKE, but they mean different things.

In Postgres, LIKE is case sensitive (i.e. ‘foo’ equals ‘foo’, but ‘foo’ doesn’t equal ‘Foo’).

In SQLite, LIKE is case insensitive for ASCII characters (i.e. ‘foo’ equals ‘Foo’). But not for non-ASCII characters. To learn more, see the docs:

https://sqlite.org/lang_expr.html#the_like_glob_regexp_and_match_operators

value_type¶: alias of int

Aggregate functions¶

Count¶

class piccolo.query.methods.select.Count(column: Optional[Column] = None, distinct: Optional[Sequence[Column]] = None, alias: str = 'count')¶

Used in Select queries, usually in conjunction with the group_by clause:

>>> await Band.select(
...     Band.manager.name.as_alias('manager_name'),
...     Count(alias='band_count')
... ).group_by(Band.manager)
[{'manager_name': 'Guido', 'count': 1}, ...]

It can also be used without the group_by clause (though you may prefer to the Table.count method instead, as it’s more convenient):

>>> await Band.select(Count())
[{'count': 3}]

Parameters

column – If specified, the count is for non-null values in that column.
distinct – If specified, the count is for distinct values in those columns.

alias –

The name of the value in the response:

# These two are equivalent:

await Band.select(
    Band.name, Count(alias="total")
).group_by(Band.name)

await Band.select(
    Band.name,
    Count().as_alias("total")
).group_by(Band.name)

Refresh¶

class piccolo.query.methods.refresh.Refresh(instance: Table, columns: t.Optional[t.Sequence[Column]] = None)¶

Used to refresh Table instances with the latest data data from the database. Accessible via refresh.

Parameters

instance – The instance to refresh.
columns – Which columns to refresh - it not specified, then all columns are refreshed.

async run(in_pool: bool = True, node: t.Optional[str] = None) → Table¶

Run it asynchronously. For example:

await my_instance.refresh().run()

# or for convenience:
await my_instance.refresh()

Modifies the instance in place, but also returns it as a convenience.

run_sync(*args, **kwargs) → Table¶

Run it synchronously. For example:

my_instance.refresh().run_sync()

LazyTableReference¶

class piccolo.columns.LazyTableReference(table_class_name: str, app_name: Optional[str] = None, module_path: Optional[str] = None)¶

Holds a reference to a Table subclass. Used to avoid circular dependencies in the references argument of ForeignKey columns.

Parameters

table_class_name – The name of the Table subclass. For example, 'Manager'.
app_name – If specified, the Table subclass is imported from a Piccolo app with the given name.
module_path – If specified, the Table subclass is imported from this path. For example, 'my_app.tables'.

Enums¶

Foreign Keys¶

class piccolo.columns.OnDelete(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)¶

Used by ForeignKey to specify the behaviour when a related row is deleted.

cascade = 'CASCADE'¶

no_action = 'NO ACTION'¶

restrict = 'RESTRICT'¶

set_default = 'SET DEFAULT'¶

set_null = 'SET NULL'¶

class piccolo.columns.OnUpdate(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)¶

Used by ForeignKey to specify the behaviour when a related row is updated.

cascade = 'CASCADE'¶

no_action = 'NO ACTION'¶

restrict = 'RESTRICT'¶

set_default = 'SET DEFAULT'¶

set_null = 'SET NULL'¶

Indexes¶

class piccolo.columns.indexes.IndexMethod(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)¶

Used to specify the index method for a Column.

btree = 'btree'¶

gin = 'gin'¶

gist = 'gist'¶

hash = 'hash'¶

Column defaults¶

Date¶

class piccolo.columns.defaults.DateOffset(days: int)¶

This makes the default value for a Date column the current date, but offset by a number of days.

For example, if you wanted the default to be tomorrow, you can specify DateOffset(days=1):

class DiscountCode(Table):
    expires = Date(default=DateOffset(days=1))

Parameters: days – The number of days to offset.

Testing¶

ModelBuilder¶

class piccolo.testing.model_builder.ModelBuilder¶

async classmethod build(table_class: Type[TableInstance], defaults: Dict[Union[Column, str], Any] = None, persist: bool = True, minimal: bool = False) → TableInstance¶

Build a Table instance with random data and save async. If the Table has any foreign keys, then the related rows are also created automatically.

Parameters

table_class – Table class to randomize.
defaults – Any values specified here will be used instead of random values.
persist – Whether to save the new instance in the database.
minimal – If True then any columns with null=True are assigned a value of None.

Examples:

# Create a new instance with all random values:
manager = await ModelBuilder.build(Manager)

# Create a new instance, with certain defaults:
manager = await ModelBuilder.build(
    Manager,
    {Manager.name: 'Guido'}
)

# Create a new instance, but don't save it in the database:
manager = await ModelBuilder.build(Manager, persist=False)

# Create a new instance, with all null values set to None:
manager = await ModelBuilder.build(Manager, minimal=True)

# We can pass other table instances in as default values:
band = await ModelBuilder.build(Band, {Band.manager: manager})

classmethod build_sync(table_class: Type[TableInstance], defaults: Dict[Union[Column, str], Any] = None, persist: bool = True, minimal: bool = False) → TableInstance¶: A sync wrapper around build().

create_db_tables / drop_db_tables¶

async piccolo.table.create_db_tables(*tables: Type[Table], if_not_exists: bool = False) → None¶

Creates the database table for each Table class passed in. The tables are created in the correct order, based on their foreign keys.

Parameters

tables – The tables to create in the database.
if_not_exists – No errors will be raised if any of the tables already exist in the database.

piccolo.table.create_db_tables_sync(*tables: Type[Table], if_not_exists: bool = False) → None¶: A sync wrapper around create_db_tables().

async piccolo.table.drop_db_tables(*tables: Type[Table]) → None¶

Drops the database table for each Table class passed in. The tables are dropped in the correct order, based on their foreign keys.

Parameters: tables – The tables to delete from the database.

piccolo.table.drop_db_tables_sync(*tables: Type[Table]) → None¶: A sync wrapper around drop_db_tables().

band	start_date
Pythonistas	2023-01-01
Pythonistas	2023-02-03
Rustaceans	2023-01-01