dballe database functions¶
-
class
dballe.
DB
¶ DB-All.e database access.
Many methods are the same in
dballe.DB
anddballe.Transaction
. The versions indballe.DB
are implemented by automatically creating a temporary transaction and running the equivalentdballe.Transaction
method inside it.dballe.DB
objects are not constructed explicitly, but via one of theDB.connect()
orDB.connect_test()
class methods.Examples:
# Connect to a database and run a query db = dballe.DB.connect_from_file("db.sqlite") query = {"latmin": 44.0, "latmax": 45.0, "lonmin": 11.0, "lonmax": 12.0} # The result is a dballe.Cursor (dballe.CursorData in this case), which can # be iterated to get results as dict objects. with db.transaction() as tr: for row in tr.query_data(query): print(row["lat"], row["lon"], row["var"], row["variable"].format("undefined")) # Insert 2 new variables in the database db.insert_data({ "lat": 44.5, "lon": 11.4, "level": dballe.Level(1), "trange": dballe.Trange(254), "date": datetime.datetime(2013, 4, 25, 12, 0, 0), "B11101": 22.4, "B12103": 17.2, })
-
attr_insert_data
(varid: int, attrs: Dict[str, Any])¶ Insert new data attributes into the database
-
attr_insert_station
(varid: int, attrs: Dict[str, Any])¶ Insert new constant station data attributes into the database
-
attr_query_data
(varid: int) → Dict[str, Any]¶ query data attributes
-
attr_query_station
(varid: int) → Dict[str, Any]¶ query constant station data attributes
-
attr_remove_data
(varid: int, attrs: Iterable[str])¶ Remove attributes from data
-
attr_remove_station
(varid: int, attrs: Iterable[str])¶ Remove attributes from constant station data
-
connect
(url: str) → dballe.DB¶ create a DB to access a database identified by a DB-All.e URL
-
connect_from_file
(name: str) → dballe.DB¶ create a DB to access a SQLite file
-
connect_from_url
(url: str) → dballe.DB¶ create a DB to access a database identified by a DB-All.e URL (deprecated, use connect instead)
-
connect_test
() → dballe.DB¶ Create a DB for running the test suite, as configured in the test environment
-
disappear
()¶ Remove all DB-All.e tables and data from the database, if possible
-
get_default_format
() → str¶ get the default DB format
-
import_messages
(messages: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message], dballe.ImporterFile], report: str=None, import_attributes: bool=False, update_station: bool=False, overwrite: bool=False)¶ Import one or more Messages into the database.
Parameters: - messages –
- a
dballe.Message
object - a sequence or iterable of
dballe.Message
objects - a
dballe.ImporterFile
that generates a sequence ofdballe.Message
objects
- a
- report – the network name to use for importing the data. If left to None, the network is selected automatically from the message type
- import_attributes – if set to True, requests the variable attributes to also be imported.
- update_station – if set to True, station information is merged with existing one in the database. If false (default), station information is imported only when the station did not exist in the database.
- overwrite – if set to True, causes existing information already in the database to be overwritten. If false (default), trying to import a message which contains data already present in the database causes the import to fail.
- varlist – if set to a string in the same format as the varlist query parameter, only imports data whose varcode is in the list.
- messages –
-
insert_data
(record: Union[Dict[str, Any], dballe.Cursor, dballe.Data], can_replace: bool=False, can_add_stations: bool=False) → Dict[str, int]¶ Insert data values in the database
The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.
-
insert_station_data
(record: Union[Dict[str, Any], dballe.Cursor, dballe.Data], can_replace: bool=False, can_add_stations: bool=False) → Dict[str, int]¶ Insert station values in the database
The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.
-
is_url
(url: str) → bool¶ Checks if a string looks like a DB-All.e DB url
-
query_data
(query: Dict[str, Any]) → dballe.CursorData¶ Query the data in the database
Returns: a cursor to iterate the query results (see dballe.CursorDataDB
)
-
query_messages
(query: Dict[str, Any]) → dballe.CursorMessage¶ Query the database returning the matching data as Message objects
This can also be used to export messages to a file. For example:
exporter = dballe.Exporter("BUFR") with open("file.bufr", "wb") as outfile: for row in tr.query_messages(...): outfile.write(exporter.to_binary(row.message))
See:
dballe.Exporter
anddballe.CursorMessage
.
-
query_station_data
(query: Dict[str, Any]) → dballe.CursorStationData¶ Query the constant station data in the database
Returns: a cursor to iterate the query results (see dballe.CursorStationDataDB
)
-
query_stations
(query: Dict[str, Any]) → dballe.CursorStation¶ Query the stations in the database
Returns: a cursor to iterate the query results (see dballe.CursorStationDB
)
-
query_summary
(query: Dict[str, Any]) → dballe.CursorSummary¶ Query the summary of the results of a query
Returns: a cursor to iterate the query results (see dballe.CursorSummaryDB
)
-
remove
(query: Dict[str, Any])¶ Remove data variables from the database (deprecated)
-
remove_all
()¶ Remove all data from the database
-
remove_data
(query: Dict[str, Any])¶ Remove data variables from the database
-
remove_station_data
(query: Dict[str, Any])¶ Remove station variables from the database
-
reset
(repinfo_file: str=None)¶ Reset the database, removing all existing Db-All.e tables and re-creating them empty.
-
set_default_format
(format: str)¶ set the default DB format
-
transaction
(readonly: bool=False) → dballe.Transaction¶ Create a new database transaction
-
vacuum
()¶ Perform database cleanup operations
-
-
class
dballe.
Transaction
¶ DB-All.e transaction
A Transaction is used to execute DB operations in an all-or-nothing fashion. In fact, most DB methods are implemented using a short-lived temporary transaction.
You cannot have more than one active dballe.Transaction for each dballe.DB. An attempt to start a second one will result in an exception being raised. Note that dballe.DB functions like
dballe.Transaction.insert_data()
create a temporary transaction to run, and so they will also fail if a transaction is currently open. The general idea is that all database work should be done inside a transaction.Transactions run using the REPEATABLE READ isolation level of the underlying database. This usually means that modifications performed inside a transaction are not visible to other database connections until the transaction is committed. If a transaction is rolled back, all changes done with it are undone.
Transactions can also be used as context managers, like this:
with db.transaction() as t: for i in range(10): t.insert({ "lat": 44.5 + i, "lon": 11.4 + i, "level": (1,), "trange": (254,), "date": datetime.datetime(2013, 4, 25, 12, 0, 0), "B11101": 22.4 + i, "B12103": 17.2 })
The dballe.Transaction methods are the same as those in dballe.DB. The version in dballe.DB is implemented by automatically creating a temporary transaction and running the dballe.Transaction method inside it.
-
attr_insert_data
(varid: int, attrs: Dict[str, Any])¶ Insert new data attributes into the database
-
attr_insert_station
(varid: int, attrs: Dict[str, Any])¶ Insert new constant station data attributes into the database
-
attr_query_data
(varid: int) → Dict[str, Any]¶ query data attributes
-
attr_query_station
(varid: int) → Dict[str, Any]¶ query constant station data attributes
-
attr_remove_data
(varid: int, attrs: Iterable[str])¶ Remove attributes from data
-
attr_remove_station
(varid: int, attrs: Iterable[str])¶ Remove attributes from constant station data
-
commit
()¶ commit the transaction
-
import_messages
(messages: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message], dballe.ImporterFile], report: str=None, import_attributes: bool=False, update_station: bool=False, overwrite: bool=False)¶ Import one or more Messages into the database.
Parameters: - messages –
- a
dballe.Message
object - a sequence or iterable of
dballe.Message
objects - a
dballe.ImporterFile
that generates a sequence ofdballe.Message
objects
- a
- report – the network name to use for importing the data. If left to None, the network is selected automatically from the message type
- import_attributes – if set to True, requests the variable attributes to also be imported.
- update_station – if set to True, station information is merged with existing one in the database. If false (default), station information is imported only when the station did not exist in the database.
- overwrite – if set to True, causes existing information already in the database to be overwritten. If false (default), trying to import a message which contains data already present in the database causes the import to fail.
- varlist – if set to a string in the same format as the varlist query parameter, only imports data whose varcode is in the list.
- messages –
-
insert_data
(record: Union[Dict[str, Any], dballe.Cursor, dballe.Data], can_replace: bool=False, can_add_stations: bool=False) → Dict[str, int]¶ Insert data values in the database
The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.
-
insert_station_data
(record: Union[Dict[str, Any], dballe.Cursor, dballe.Data], can_replace: bool=False, can_add_stations: bool=False) → Dict[str, int]¶ Insert station values in the database
The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.
-
query_data
(query: Dict[str, Any]) → dballe.CursorData¶ Query the data in the database
Returns: a cursor to iterate the query results (see dballe.CursorDataDB
)
-
query_messages
(query: Dict[str, Any]) → dballe.CursorMessage¶ Query the database returning the matching data as Message objects
This can also be used to export messages to a file. For example:
exporter = dballe.Exporter("BUFR") with open("file.bufr", "wb") as outfile: for row in tr.query_messages(...): outfile.write(exporter.to_binary(row.message))
See:
dballe.Exporter
anddballe.CursorMessage
.
-
query_station_data
(query: Dict[str, Any]) → dballe.CursorStationData¶ Query the constant station data in the database
Returns: a cursor to iterate the query results (see dballe.CursorStationDataDB
)
-
query_stations
(query: Dict[str, Any]) → dballe.CursorStation¶ Query the stations in the database
Returns: a cursor to iterate the query results (see dballe.CursorStationDB
)
-
query_summary
(query: Dict[str, Any]) → dballe.CursorSummary¶ Query the summary of the results of a query
Returns: a cursor to iterate the query results (see dballe.CursorSummaryDB
)
-
remove
(query: Dict[str, Any])¶ Remove data variables from the database (deprecated)
-
remove_all
()¶ Remove all data from the database
-
remove_data
(query: Dict[str, Any])¶ Remove data variables from the database
-
remove_station_data
(query: Dict[str, Any])¶ Remove station variables from the database
-
rollback
()¶ roll back the transaction
-
-
class
dballe.
Explorer
¶ Browser for a summary of DB-All-e database of message contents
-
all_levels
¶ get all level values
-
all_reports
¶ get all report values
-
all_stations
¶ get all stations
-
all_stats
¶ get the stats for all values
-
all_tranges
¶ get all time range values
-
all_varcodes
¶ get all varcode values
-
levels
¶ get all the level values currently selected
-
query_summary
() → dballe.CursorSummarySummary¶ Get the currently selected Explorer summary information
Returns: a cursor to iterate the query results (see dballe.CursorSummarySummary
)
-
query_summary_all
() → dballe.CursorSummarySummary¶ Get all the Explorer summary information.
Returns: a cursor to iterate the query results (see dballe.CursorSummarySummary
)
-
rebuild
()¶ Empty the Explorer and start adding new data to it.
Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.
-
reports
¶ get all the report values currently selected
-
set_filter
()¶ Set a new filter, updating all browsing data
-
stations
¶ get all the stations currently selected
-
stats
¶ get stats for the currently selected values
-
to_json
()¶ Serialize the contents of this explorer to JSON.
Only the global summary is serialized: the current query is not preserved.
-
tranges
¶ get all the time range values currently selected
-
update
()¶ Start adding new data to the Explorer without clearing it first.
Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.
-
varcodes
¶ get all the varcode values currently selected
-
-
class
dballe.
DBExplorer
¶ Browser for a summary of DB-All-e database of message contents
-
all_levels
¶ get all level values
-
all_reports
¶ get all report values
-
all_stations
¶ get all stations
-
all_stats
¶ get the stats for all values
-
all_tranges
¶ get all time range values
-
all_varcodes
¶ get all varcode values
-
levels
¶ get all the level values currently selected
-
query_summary
() → dballe.CursorSummaryDBSummary¶ Get the currently selected Explorer summary information
Returns: a cursor to iterate the query results (see dballe.CursorSummaryDBSummary
)
-
query_summary_all
() → dballe.CursorSummaryDBSummary¶ Get all the Explorer summary information.
Returns: a cursor to iterate the query results (see dballe.CursorSummaryDBSummary
)
-
rebuild
()¶ Empty the Explorer and start adding new data to it.
Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.
-
reports
¶ get all the report values currently selected
-
set_filter
()¶ Set a new filter, updating all browsing data
-
stations
¶ get all the stations currently selected
-
stats
¶ get stats for the currently selected values
-
to_json
()¶ Serialize the contents of this explorer to JSON.
Only the global summary is serialized: the current query is not preserved.
-
tranges
¶ get all the time range values currently selected
-
update
()¶ Start adding new data to the Explorer without clearing it first.
Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.
-
varcodes
¶ get all the varcode values currently selected
-
-
class
dballe.
ExplorerUpdate
¶ Manage updates to an Explorer
-
add_db
()¶ Add the summary of the contents of the given database to the Explorer.
-
add_explorer
()¶ Add the contents of the given Explorer or DBExplorer to the Explorer.
-
add_json
()¶ Add the contents of the given JSON string to the Explorer.
-
add_messages
()¶ Add dballe.Message objects to the explorer.
It takes the same messages argument of dballe.DB.import_messages
-
-
class
dballe.
DBExplorerUpdate
¶ Manage updates to an Explorer
-
add_db
()¶ Add the summary of the contents of the given database to the Explorer.
-
add_explorer
()¶ Add the contents of the given Explorer or DBExplorer to the Explorer.
-
add_json
()¶ Add the contents of the given JSON string to the Explorer.
-
add_messages
()¶ Add dballe.Message objects to the explorer.
It takes the same messages argument of dballe.DB.import_messages
-