Unofficial git mirror of SQLite sources (see link for build instructions)

Related tags

Database sqlite
Overview

SQLite Source Repository

This repository contains the complete source code for the SQLite database engine. Some test scripts are also included. However, many other test scripts and most of the documentation are managed separately.

Version Control

SQLite sources are managed using the Fossil, a distributed version control system that was specifically designed and written to support SQLite development. The Fossil repository contains the urtext.

If you are reading this on GitHub or some other Git repository or service, then you are looking at a mirror. The names of check-ins and other artifacts in a Git mirror are different from the official names for those objects. The offical names for check-ins are found in a footer on the check-in comment for authorized mirrors. The official check-in name can also be seen in the manifest.uuid file in the root of the tree. Always use the official name, not the Git-name, when communicating about an SQLite check-in.

If you pulled your SQLite source code from a secondary source and want to verify its integrity, there are hints on how to do that in the Verifying Code Authenticity section below.

Obtaining The Code

If you do not want to use Fossil, you can download tarballs or ZIP archives or SQLite archives as follows:

  • Lastest trunk check-in as Tarball, ZIP-archive, or SQLite-archive.

  • Latest release as Tarball, ZIP-archive, or SQLite-archive.

  • For other check-ins, substitute an appropriate branch name or tag or hash prefix in place of "release" in the URLs of the previous bullet. Or browse the timeline to locate the check-in desired, click on its information page link, then click on the "Tarball" or "ZIP Archive" links on the information page.

If you do want to use Fossil to check out the source tree, first install Fossil version 2.0 or later. (Source tarballs and precompiled binaries available here. Fossil is a stand-alone program. To install, simply download or build the single executable file and put that file someplace on your $PATH.) Then run commands like this:

    mkdir -p ~/sqlite ~/Fossils
    cd ~/sqlite
    fossil clone https://www.sqlite.org/src ~/Fossils/sqlite.fossil
    fossil open ~/Fossils/sqlite.fossil

After setting up a repository using the steps above, you can always update to the lastest version using:

    fossil update trunk   ;# latest trunk check-in
    fossil update release ;# latest official release

Or type "fossil ui" to get a web-based user interface.

Compiling for Unix-like systems

First create a directory in which to place the build products. It is recommended, but not required, that the build directory be separate from the source directory. Cd into the build directory and then from the build directory run the configure script found at the root of the source tree. Then run "make".

For example:

    tar xzf sqlite.tar.gz    ;#  Unpack the source tree into "sqlite"
    mkdir bld                ;#  Build will occur in a sibling directory
    cd bld                   ;#  Change to the build directory
    ../sqlite/configure      ;#  Run the configure script
    make                     ;#  Run the makefile.
    make sqlite3.c           ;#  Build the "amalgamation" source file
    make test                ;#  Run some tests (requires Tcl)

See the makefile for additional targets.

The configure script uses autoconf 2.61 and libtool. If the configure script does not work out for you, there is a generic makefile named "Makefile.linux-gcc" in the top directory of the source tree that you can copy and edit to suit your needs. Comments on the generic makefile show what changes are needed.

Using MSVC for Windows systems

On Windows, all applicable build products can be compiled with MSVC. First open the command prompt window associated with the desired compiler version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE with the provided "Makefile.msc" to build one of the supported targets.

For example, from the parent directory of the source subtree named "sqlite":

    mkdir bld
    cd bld
    nmake /f ..\sqlite\Makefile.msc TOP=..\sqlite
    nmake /f ..\sqlite\Makefile.msc sqlite3.c TOP=..\sqlite
    nmake /f ..\sqlite\Makefile.msc sqlite3.dll TOP=..\sqlite
    nmake /f ..\sqlite\Makefile.msc sqlite3.exe TOP=..\sqlite
    nmake /f ..\sqlite\Makefile.msc test TOP=..\sqlite

There are several build options that can be set via the NMAKE command line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument to the "sqlite3.dll" command line above. When debugging into the SQLite code, adding the "DEBUG=1" argument to one of the above command lines is recommended.

SQLite does not require Tcl to run, but a Tcl installation is required by the makefiles (including those for MSVC). SQLite contains a lot of generated code and Tcl is used to do much of that code generation.

Source Code Tour

Most of the core source files are in the src/ subdirectory. The src/ folder also contains files used to build the "testfixture" test harness. The names of the source files used by "testfixture" all begin with "test". The src/ also contains the "shell.c" file which is the main program for the "sqlite3.exe" command-line shell and the "tclsqlite.c" file which implements the Tcl bindings for SQLite. (Historical note: SQLite began as a Tcl extension and only later escaped to the wild as an independent library.)

Test scripts and programs are found in the test/ subdirectory. Addtional test code is found in other source repositories. See How SQLite Is Tested for additional information.

The ext/ subdirectory contains code for extensions. The Full-text search engine is in ext/fts3. The R-Tree engine is in ext/rtree. The ext/misc subdirectory contains a number of smaller, single-file extensions, such as a REGEXP operator.

The tool/ subdirectory contains various scripts and programs used for building generated source code files or for testing or for generating accessory programs such as "sqlite3_analyzer(.exe)".

Generated Source Code Files

Several of the C-language source files used by SQLite are generated from other sources rather than being typed in manually by a programmer. This section will summarize those automatically-generated files. To create all of the automatically-generated files, simply run "make target_source". The "target_source" make target will create a subdirectory "tsrc/" and fill it with all the source files needed to build SQLite, both manually-edited files and automatically-generated files.

The SQLite interface is defined by the sqlite3.h header file, which is generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The Tcl script at tool/mksqlite3h.tcl does the conversion. The manifest.uuid file contains the SHA3 hash of the particular check-in and is used to generate the SQLITE_SOURCE_ID macro. The VERSION file contains the current SQLite version number. The sqlite3.h header is really just a copy of src/sqlite.h.in with the source-id and version number inserted at just the right spots. Note that comment text in the sqlite3.h file is used to generate much of the SQLite API documentation. The Tcl scripts used to generate that documentation are in a separate source repository.

The SQL language parser is parse.c which is generate from a grammar in the src/parse.y file. The conversion of "parse.y" into "parse.c" is done by the lemon LALR(1) parser generator. The source code for lemon is at tool/lemon.c. Lemon uses the tool/lempar.c file as a template for generating its parser. Lemon also generates the parse.h header file, at the same time it generates parse.c.

The opcodes.h header file contains macros that define the numbers corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h file is generated by the scanning the src/vdbe.c source file. The Tcl script at ./mkopcodeh.tcl does this scan and generates opcodes.h. A second Tcl script, ./mkopcodec.tcl, then scans opcodes.h to generate the opcodes.c source file, which contains a reverse mapping from opcode-number to opcode-name that is used for EXPLAIN output.

The keywordhash.h header file contains the definition of a hash table that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into the numeric codes used by the parse.c parser. The keywordhash.h file is generated by a C-language program at tool mkkeywordhash.c.

The pragma.h header file contains various definitions used to parse and implement the PRAGMA statements. The header is generated by a script tool/mkpragmatab.tcl. If you want to add a new PRAGMA, edit the tool/mkpragmatab.tcl file to insert the information needed by the parser for your new PRAGMA, then run the script to regenerate the pragma.h header file.

The Amalgamation

All of the individual C source code and header files (both manually-edited and automatically-generated) can be combined into a single big source file sqlite3.c called "the amalgamation". The amalgamation is the recommended way of using SQLite in a larger application. Combining all individual source code files into a single big source code file allows the C compiler to perform more cross-procedure analysis and generate better code. SQLite runs about 5% faster when compiled from the amalgamation versus when compiled from individual source files.

The amalgamation is generated from the tool/mksqlite3c.tcl Tcl script. First, all of the individual source files must be gathered into the tsrc/ subdirectory (using the equivalent of "make target_source") then the tool/mksqlite3c.tcl script is run to copy them all together in just the right order while resolving internal "#include" references.

The amalgamation source file is more than 200K lines long. Some symbolic debuggers (most notably MSVC) are unable to deal with files longer than 64K lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl, can be run on the amalgamation to break it up into a single small C file called sqlite3-all.c that does #include on about seven other files named sqlite3-1.c, sqlite3-2.c, ..., sqlite3-7.c. In this way, all of the source code is contained within a single translation unit so that the compiler can do extra cross-procedure optimization, but no individual source file exceeds 32K lines in length.

How It All Fits Together

SQLite is modular in design. See the architectural description for details. Other documents that are useful in (helping to understand how SQLite works include the file format description, the virtual machine that runs prepared statements, the description of how transactions work, and the overview of the query planner.

Years of effort have gone into optimizating SQLite, both for small size and high performance. And optimizations tend to result in complex code. So there is a lot of complexity in the current SQLite implementation. It will not be the easiest library in the world to hack.

Key files:

  • sqlite.h.in - This file defines the public interface to the SQLite library. Readers will need to be familiar with this interface before trying to understand how the library works internally.

  • sqliteInt.h - this header file defines many of the data objects used internally by SQLite. In addition to "sqliteInt.h", some subsystems have their own header files.

  • parse.y - This file describes the LALR(1) grammar that SQLite uses to parse SQL statements, and the actions that are taken at each step in the parsing process.

  • vdbe.c - This file implements the virtual machine that runs prepared statements. There are various helper files whose names begin with "vdbe". The VDBE has access to the vdbeInt.h header file which defines internal data objects. The rest of SQLite interacts with the VDBE through an interface defined by vdbe.h.

  • where.c - This file (together with its helper files named by "where*.c") analyzes the WHERE clause and generates virtual machine code to run queries efficiently. This file is sometimes called the "query optimizer". It has its own private header file, whereInt.h, that defines data objects used internally.

  • btree.c - This file contains the implementation of the B-Tree storage engine used by SQLite. The interface to the rest of the system is defined by "btree.h". The "btreeInt.h" header defines objects used internally by btree.c and not published to the rest of the system.

  • pager.c - This file contains the "pager" implementation, the module that implements transactions. The "pager.h" header file defines the interface between pager.c and the rest of the system.

  • os_unix.c and os_win.c - These two files implement the interface between SQLite and the underlying operating system using the run-time pluggable VFS interface.

  • shell.c.in - This file is not part of the core SQLite library. This is the file that, when linked against sqlite3.a, generates the "sqlite3.exe" command-line shell. The "shell.c.in" file is transformed into "shell.c" as part of the build process.

  • tclsqlite.c - This file implements the Tcl bindings for SQLite. It is not part of the core SQLite library. But as most of the tests in this repository are written in Tcl, the Tcl language bindings are important.

  • test.c* - Files in the src/ folder that begin with "test" go into building the "testfixture.exe" program. The testfixture.exe program is an enhanced Tcl shell. The testfixture.exe program runs scripts in the test/ folder to validate the core SQLite code. The testfixture program (and some other test programs too) is build and run when you type "make test".

  • ext/misc/json1.c - This file implements the various JSON functions that are build into SQLite.

There are many other source files. Each has a succinct header comment that describes its purpose and role within the larger system.

Verifying Code Authenticity

The manifest file at the root directory of the source tree contains either a SHA3-256 hash (for newer files) or a SHA1 hash (for older files) for every source file in the repository. The name of the version of the entire source tree is just the SHA3-256 hash of the manifest file itself, possibly with the last line of that file omitted if the last line begins with "# Remove this line". The manifest.uuid file should contain the SHA3-256 hash of the manifest file. If all of the above hash comparisons are correct, then you can be confident that your source tree is authentic and unadulterated.

The format of the manifest file should be mostly self-explanatory, but if you want details, they are available here.

Contacts

The main SQLite website is http://www.sqlite.org/ with geographically distributed backups at http://www2.sqlite.org/ and http://www3.sqlite.org/.

You might also like...
Verneuil is a VFS extension for SQLite that asynchronously replicates databases to S3-compatible blob stores.
Verneuil is a VFS extension for SQLite that asynchronously replicates databases to S3-compatible blob stores.

Verneuil: streaming replication for sqlite Verneuil1 [vɛʁnœj] is a VFS (OS abstraction layer) for sqlite that accesses local database files like the d

C++ ORM for SQLite

Hiberlite ORM C++ object-relational mapping with API inspired by the awesome Boost.Serialization - that means almost no API to learn. Usage Just compi

The C++14 wrapper around sqlite library

sqlite modern cpp wrapper This library is a lightweight modern wrapper around sqlite C api . #includeiostream #include sqlite_modern_cpp.h using n

A hook for Project Zomboid that intercepts files access for savegames and puts them in an SQLite DB instead.

ZomboidDB This project consists of a library and patcher that results in file calls for your savegame(s) being transparently intercepted and redirecte

Lightweight C++ wrapper for SQLite

NLDatabase Lightweight C++ wrapper for SQLite. Requirements C++11 compiler SQLite 3 Usage Let's open a database file and read some rows: #include "NLD

Writing a sqlite clone from scratch in C++

如何用C++实现一个简易数据库 基于cstack/db_tutorial C语言版本 KCNyu 2022/2/2 作为笔者写的第一个系列型教程,还是选择基于前人的教程经验以及添加一些自己个人的探索。也许有很多纰漏之处,希望大家指正。 1. 数据库是什么? 数据库是“按照数据结构来组织、存储和管理数

Libav github mirror, clone of git://git.libav.org/libav

Libav Libav is a collection of libraries and tools to process multimedia content such as audio, video, subtitles and related metadata. Libraries libav

DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files compatible with SQLite.
DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files compatible with SQLite.

DB Browser for SQLite What it is DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files c

React-native-quick-sqlite - ⚡️ The fastest SQLite implementation for react-native.
React-native-quick-sqlite - ⚡️ The fastest SQLite implementation for react-native.

React Native Quick SQLite The **fastest** SQLite implementation for react-native. Copy typeORM patch-package from example dir npm i react-nati

Sysmon For Linux install and build instructions

The packages are available in the official Microsoft Linux repositories and instructions on how to install the packages for the different Linux distributions can be found in the Installation instructions.

The official mirror of the V8 Git repository

V8 JavaScript Engine V8 is Google's open source JavaScript engine. V8 implements ECMAScript as specified in ECMA-262. V8 is written in C++ and is used

Cpp-btree - Git mirror of the official (mercurial) repository of cpp-btree

This library is a C++ template library and, as such, there is no library to build and install. Copy the .h files and use them! See http://code.googl

ncurses Git mirror

------------------------------------------------------------------------------- -- Copyright 2020,2021 Thomas E. Dickey

wl-mirror - a simple Wayland output mirror client
wl-mirror - a simple Wayland output mirror client

wl-mirror - a simple Wayland output mirror client wl-mirror attempts to provide a solution to sway's lack of output mirroring by mirroring an output o

How To Build The NeXT ROM Monitor From Sources

How To Build The NeXT ROM Monitor From Sources

Automated upstream mirror for bpftool stand-alone build.

bpftool This is a mirror of bpf-next Linux source tree's tools/bpf/bpftool directory, plus its few dependencies from under kernel/bpf/, and its suppor

Learn how to connect your Flexispot (LoctekMotion) desk to the internet. This repository contains a collection of scripts to get your started, combined with research and instructions.
Learn how to connect your Flexispot (LoctekMotion) desk to the internet. This repository contains a collection of scripts to get your started, combined with research and instructions.

(image source: Windows Central) Turn your LoctekMotion/FlexiSpot desk into a smart desk Recently I acquired a new standing desk from FlexiSpot. During

Comments
  • Cumulative changes to support parsing NUL characters

    Cumulative changes to support parsing NUL characters

    Adds pragma allow_inline_nul which takes a simple boolean parameter =on/off to enable. Enabling pragma Allows specifying the length of the statement in prepared statements, without requiring binding.
    Adds sqlite3_sql_utf8() and sqlite3_expanded_sql_utf8() functions to be able to get a statement which has NUL characters in it, or has parameters with NUL characters in them bound to a parameter. Adds test test/inlinenul.c to test basic functionality. Has a slight improvement in speed for most tests.

    opened by d3x0r 2
  • archive: -Au now updates only newer files

    archive: -Au now updates only newer files

    I have got a directory with more then 50000 files. Actualy this is a Golang GOPATH directory. By default

    ./sqlite3.exe ./testdb.db -AvuC d:/Go GOPATH takes too much time trying to update every file in archive db. With this proposal updating sqlar archive with many files now takes much less time.

    opened by zavla 1
React-native-quick-sqlite - ⚡️ The fastest SQLite implementation for react-native.

React Native Quick SQLite The **fastest** SQLite implementation for react-native. Copy typeORM patch-package from example dir npm i react-nati

Oscar Franco 397 Nov 21, 2022
❤️ SQLite ORM light header only library for modern C++

SQLite ORM SQLite ORM light header only library for modern C++ Status Branch Travis Appveyor master dev Advantages No raw string queries Intuitive syn

Yevgeniy Zakharov 1.7k Nov 19, 2022
SQLean: all the missing SQLite functions

SQLite has very few functions compared to other DBMS. SQLite authors see this as a feature rather than a bug, because SQLite has extension mechanism in place.

Anton Zhiyanov 1.5k Nov 16, 2022
An SQLite binding for node.js with built-in encryption, focused on simplicity and (async) performance

Description An SQLite (more accurately SQLite3MultipleCiphers) binding for node.js focused on simplicity and (async) performance. When dealing with en

mscdex 14 May 15, 2022
Yet another SQLite wrapper for Nim

Yet another SQLite wrapper for Nim Features: Design for ARC/ORC, you don’t need to close the connection manually Use importdb macro to create helper f

Code Hz 8 Jul 4, 2022
SpDB is a data integration tool designed to organize scientific data from different sources under the same namespace according to a global schema and to provide access to them in a unified form (views)

SpDB is a data integration tool designed to organize scientific data from different sources under the same namespace according to a global schema and to provide access to them in a unified form (views). Its main purpose is to provide a unified data access interface for complex scientific computations in order to enable the interaction and integration between different programs and databases.

YU Zhi 0 Jun 22, 2022
A friendly and lightweight C++ database library for MySQL, PostgreSQL, SQLite and ODBC.

QTL QTL is a C ++ library for accessing SQL databases and currently supports MySQL, SQLite, PostgreSQL and ODBC. QTL is a lightweight library that con

null 167 Nov 22, 2022
Fork of sqlite4java with updated SQLite and very basic compiler hardening enabled.

Download latest version: sqlite4java-392 with SQLite 3.8.7, Windows/Linux/Mac OS X/Android binaries OSGi bundle 1.0.392 with sqlite4java-392 Files for

GrapheneOS 6 Oct 26, 2022
An updated fork of sqlite_protobuf, a SQLite extension for extracting values from serialized Protobuf messages.

This fork of sqlite_protobuf fixes some issues (e.g., #15) and removes the test suite that we do not use. It also comes with proto_table, a C library

Backtrace Labs 18 Oct 19, 2022
Serverless SQLite database read from and write to Object Storage Service, run on FaaS platform.

serverless-sqlite Serverless SQLite database read from and write to Object Storage Service, run on FaaS platform. NOTES: This repository is still in t

老雷 7 May 12, 2022