1<h1 align="center">SQLite Source Repository</h1> 2 3This repository contains the complete source code for the 4[SQLite database engine](https://sqlite.org/). Some test scripts 5are also included. However, many other test scripts 6and most of the documentation are managed separately. 7 8## Version Control 9 10SQLite sources are managed using the 11[Fossil](https://www.fossil-scm.org/), a distributed version control system 12that was specifically designed and written to support SQLite development. 13The [Fossil repository](https://sqlite.org/src/timeline) contains the urtext. 14 15If you are reading this on GitHub or some other Git repository or service, 16then you are looking at a mirror. The names of check-ins and 17other artifacts in a Git mirror are different from the official 18names for those objects. The official names for check-ins are 19found in a footer on the check-in comment for authorized mirrors. 20The official check-in name can also be seen in the `manifest.uuid` file 21in the root of the tree. Always use the official name, not the 22Git-name, when communicating about an SQLite check-in. 23 24If you pulled your SQLite source code from a secondary source and want to 25verify its integrity, there are hints on how to do that in the 26[Verifying Code Authenticity](#vauth) section below. 27 28## Obtaining The Code 29 30If you do not want to use Fossil, you can download tarballs or ZIP 31archives or [SQLite archives](https://sqlite.org/cli.html#sqlar) as follows: 32 33 * Latest trunk check-in as 34 [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz), 35 [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip), or 36 [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar). 37 38 * Latest release as 39 [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release), 40 [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip?r=release), or 41 [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar?r=release). 42 43 * For other check-ins, substitute an appropriate branch name or 44 tag or hash prefix in place of "release" in the URLs of the previous 45 bullet. Or browse the [timeline](https://www.sqlite.org/src/timeline) 46 to locate the check-in desired, click on its information page link, 47 then click on the "Tarball" or "ZIP Archive" links on the information 48 page. 49 50If you do want to use Fossil to check out the source tree, 51first install Fossil version 2.0 or later. 52(Source tarballs and precompiled binaries available 53[here](https://www.fossil-scm.org/fossil/uv/download.html). Fossil is 54a stand-alone program. To install, simply download or build the single 55executable file and put that file someplace on your $PATH.) 56Then run commands like this: 57 58 mkdir -p ~/sqlite ~/Fossils 59 cd ~/sqlite 60 fossil clone https://www.sqlite.org/src ~/Fossils/sqlite.fossil 61 fossil open ~/Fossils/sqlite.fossil 62 63After setting up a repository using the steps above, you can always 64update to the latest version using: 65 66 fossil update trunk ;# latest trunk check-in 67 fossil update release ;# latest official release 68 69Or type "fossil ui" to get a web-based user interface. 70 71## Compiling for Unix-like systems 72 73First create a directory in which to place 74the build products. It is recommended, but not required, that the 75build directory be separate from the source directory. Cd into the 76build directory and then from the build directory run the configure 77script found at the root of the source tree. Then run "make". 78 79For example: 80 81 tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite" 82 mkdir bld ;# Build will occur in a sibling directory 83 cd bld ;# Change to the build directory 84 ../sqlite/configure ;# Run the configure script 85 make ;# Run the makefile. 86 make sqlite3.c ;# Build the "amalgamation" source file 87 make test ;# Run some tests (requires Tcl) 88 89See the makefile for additional targets. 90 91The configure script uses autoconf 2.61 and libtool. If the configure 92script does not work out for you, there is a generic makefile named 93"Makefile.linux-gcc" in the top directory of the source tree that you 94can copy and edit to suit your needs. Comments on the generic makefile 95show what changes are needed. 96 97## Using MSVC for Windows systems 98 99On Windows, all applicable build products can be compiled with MSVC. 100First open the command prompt window associated with the desired compiler 101version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE 102with the provided "Makefile.msc" to build one of the supported targets. 103 104For example, from the parent directory of the source subtree named "sqlite": 105 106 mkdir bld 107 cd bld 108 nmake /f ..\sqlite\Makefile.msc TOP=..\sqlite 109 nmake /f ..\sqlite\Makefile.msc sqlite3.c TOP=..\sqlite 110 nmake /f ..\sqlite\Makefile.msc sqlite3.dll TOP=..\sqlite 111 nmake /f ..\sqlite\Makefile.msc sqlite3.exe TOP=..\sqlite 112 nmake /f ..\sqlite\Makefile.msc test TOP=..\sqlite 113 114There are several build options that can be set via the NMAKE command 115line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument 116to the "sqlite3.dll" command line above. When debugging into the SQLite 117code, adding the "DEBUG=1" argument to one of the above command lines is 118recommended. 119 120SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation 121is required by the makefiles (including those for MSVC). SQLite contains 122a lot of generated code and Tcl is used to do much of that code generation. 123 124## Source Code Tour 125 126Most of the core source files are in the **src/** subdirectory. The 127**src/** folder also contains files used to build the "testfixture" test 128harness. The names of the source files used by "testfixture" all begin 129with "test". 130The **src/** also contains the "shell.c" file 131which is the main program for the "sqlite3.exe" 132[command-line shell](https://sqlite.org/cli.html) and 133the "tclsqlite.c" file which implements the 134[Tcl bindings](https://sqlite.org/tclsqlite.html) for SQLite. 135(Historical note: SQLite began as a Tcl 136extension and only later escaped to the wild as an independent library.) 137 138Test scripts and programs are found in the **test/** subdirectory. 139Additional test code is found in other source repositories. 140See [How SQLite Is Tested](http://www.sqlite.org/testing.html) for 141additional information. 142 143The **ext/** subdirectory contains code for extensions. The 144Full-text search engine is in **ext/fts3**. The R-Tree engine is in 145**ext/rtree**. The **ext/misc** subdirectory contains a number of 146smaller, single-file extensions, such as a REGEXP operator. 147 148The **tool/** subdirectory contains various scripts and programs used 149for building generated source code files or for testing or for generating 150accessory programs such as "sqlite3_analyzer(.exe)". 151 152### Generated Source Code Files 153 154Several of the C-language source files used by SQLite are generated from 155other sources rather than being typed in manually by a programmer. This 156section will summarize those automatically-generated files. To create all 157of the automatically-generated files, simply run "make target_source". 158The "target_source" make target will create a subdirectory "tsrc/" and 159fill it with all the source files needed to build SQLite, both 160manually-edited files and automatically-generated files. 161 162The SQLite interface is defined by the **sqlite3.h** header file, which is 163generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The 164[Tcl script](http://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion. 165The manifest.uuid file contains the SHA3 hash of the particular check-in 166and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file 167contains the current SQLite version number. The sqlite3.h header is really 168just a copy of src/sqlite.h.in with the source-id and version number inserted 169at just the right spots. Note that comment text in the sqlite3.h file is 170used to generate much of the SQLite API documentation. The Tcl scripts 171used to generate that documentation are in a separate source repository. 172 173The SQL language parser is **parse.c** which is generated from a grammar in 174the src/parse.y file. The conversion of "parse.y" into "parse.c" is done 175by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code 176for lemon is at tool/lemon.c. Lemon uses the tool/lempar.c file as a 177template for generating its parser. 178Lemon also generates the **parse.h** header file, at the same time it 179generates parse.c. 180 181The **opcodes.h** header file contains macros that define the numbers 182corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h 183file is generated by scanning the src/vdbe.c source file. The 184Tcl script at ./mkopcodeh.tcl does this scan and generates opcodes.h. 185A second Tcl script, ./mkopcodec.tcl, then scans opcodes.h to generate 186the **opcodes.c** source file, which contains a reverse mapping from 187opcode-number to opcode-name that is used for EXPLAIN output. 188 189The **keywordhash.h** header file contains the definition of a hash table 190that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into 191the numeric codes used by the parse.c parser. The keywordhash.h file is 192generated by a C-language program at tool mkkeywordhash.c. 193 194The **pragma.h** header file contains various definitions used to parse 195and implement the PRAGMA statements. The header is generated by a 196script **tool/mkpragmatab.tcl**. If you want to add a new PRAGMA, edit 197the **tool/mkpragmatab.tcl** file to insert the information needed by the 198parser for your new PRAGMA, then run the script to regenerate the 199**pragma.h** header file. 200 201### The Amalgamation 202 203All of the individual C source code and header files (both manually-edited 204and automatically-generated) can be combined into a single big source file 205**sqlite3.c** called "the amalgamation". The amalgamation is the recommended 206way of using SQLite in a larger application. Combining all individual 207source code files into a single big source code file allows the C compiler 208to perform more cross-procedure analysis and generate better code. SQLite 209runs about 5% faster when compiled from the amalgamation versus when compiled 210from individual source files. 211 212The amalgamation is generated from the tool/mksqlite3c.tcl Tcl script. 213First, all of the individual source files must be gathered into the tsrc/ 214subdirectory (using the equivalent of "make target_source") then the 215tool/mksqlite3c.tcl script is run to copy them all together in just the 216right order while resolving internal "#include" references. 217 218The amalgamation source file is more than 200K lines long. Some symbolic 219debuggers (most notably MSVC) are unable to deal with files longer than 64K 220lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl, 221can be run on the amalgamation to break it up into a single small C file 222called **sqlite3-all.c** that does #include on about seven other files 223named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-7.c**. In this way, 224all of the source code is contained within a single translation unit so 225that the compiler can do extra cross-procedure optimization, but no 226individual source file exceeds 32K lines in length. 227 228## How It All Fits Together 229 230SQLite is modular in design. 231See the [architectural description](http://www.sqlite.org/arch.html) 232for details. Other documents that are useful in 233(helping to understand how SQLite works include the 234[file format](http://www.sqlite.org/fileformat2.html) description, 235the [virtual machine](http://www.sqlite.org/opcode.html) that runs 236prepared statements, the description of 237[how transactions work](http://www.sqlite.org/atomiccommit.html), and 238the [overview of the query planner](http://www.sqlite.org/optoverview.html). 239 240Years of effort have gone into optimizing SQLite, both 241for small size and high performance. And optimizations tend to result in 242complex code. So there is a lot of complexity in the current SQLite 243implementation. It will not be the easiest library in the world to hack. 244 245Key files: 246 247 * **sqlite.h.in** - This file defines the public interface to the SQLite 248 library. Readers will need to be familiar with this interface before 249 trying to understand how the library works internally. 250 251 * **sqliteInt.h** - this header file defines many of the data objects 252 used internally by SQLite. In addition to "sqliteInt.h", some 253 subsystems have their own header files. 254 255 * **parse.y** - This file describes the LALR(1) grammar that SQLite uses 256 to parse SQL statements, and the actions that are taken at each step 257 in the parsing process. 258 259 * **vdbe.c** - This file implements the virtual machine that runs 260 prepared statements. There are various helper files whose names 261 begin with "vdbe". The VDBE has access to the vdbeInt.h header file 262 which defines internal data objects. The rest of SQLite interacts 263 with the VDBE through an interface defined by vdbe.h. 264 265 * **where.c** - This file (together with its helper files named 266 by "where*.c") analyzes the WHERE clause and generates 267 virtual machine code to run queries efficiently. This file is 268 sometimes called the "query optimizer". It has its own private 269 header file, whereInt.h, that defines data objects used internally. 270 271 * **btree.c** - This file contains the implementation of the B-Tree 272 storage engine used by SQLite. The interface to the rest of the system 273 is defined by "btree.h". The "btreeInt.h" header defines objects 274 used internally by btree.c and not published to the rest of the system. 275 276 * **pager.c** - This file contains the "pager" implementation, the 277 module that implements transactions. The "pager.h" header file 278 defines the interface between pager.c and the rest of the system. 279 280 * **os_unix.c** and **os_win.c** - These two files implement the interface 281 between SQLite and the underlying operating system using the run-time 282 pluggable VFS interface. 283 284 * **shell.c.in** - This file is not part of the core SQLite library. This 285 is the file that, when linked against sqlite3.a, generates the 286 "sqlite3.exe" command-line shell. The "shell.c.in" file is transformed 287 into "shell.c" as part of the build process. 288 289 * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It 290 is not part of the core SQLite library. But as most of the tests in this 291 repository are written in Tcl, the Tcl language bindings are important. 292 293 * **test*.c** - Files in the src/ folder that begin with "test" go into 294 building the "testfixture.exe" program. The testfixture.exe program is 295 an enhanced Tcl shell. The testfixture.exe program runs scripts in the 296 test/ folder to validate the core SQLite code. The testfixture program 297 (and some other test programs too) is built and run when you type 298 "make test". 299 300 * **ext/misc/json1.c** - This file implements the various JSON functions 301 that are built into SQLite. 302 303There are many other source files. Each has a succinct header comment that 304describes its purpose and role within the larger system. 305 306<a name="vauth"></a> 307## Verifying Code Authenticity 308 309The `manifest` file at the root directory of the source tree 310contains either a SHA3-256 hash (for newer files) or a SHA1 hash (for 311older files) for every source file in the repository. 312The name of the version of the entire source tree is just the 313SHA3-256 hash of the `manifest` file itself, possibly with the 314last line of that file omitted if the last line begins with 315"`# Remove this line`". 316The `manifest.uuid` file should contain the SHA3-256 hash of the 317`manifest` file. If all of the above hash comparisons are correct, then 318you can be confident that your source tree is authentic and unadulterated. 319 320The format of the `manifest` file should be mostly self-explanatory, but 321if you want details, they are available 322[here](https://fossil-scm.org/fossil/doc/trunk/www/fileformat.wiki#manifest). 323 324## Contacts 325 326The main SQLite website is [http://www.sqlite.org/](http://www.sqlite.org/) 327with geographically distributed backups at 328[http://www2.sqlite.org/](http://www2.sqlite.org) and 329[http://www3.sqlite.org/](http://www3.sqlite.org). 330