1<h1 align="center">SQLite Source Repository</h1> 2 3This repository contains the complete source code for the 4[SQLite database engine](https://sqlite.org/). Some test scripts 5are also included. However, many other test scripts 6and most of the documentation are managed separately. 7 8## Version Control 9 10SQLite sources are managed using the 11[Fossil](https://www.fossil-scm.org/), a distributed version control system 12that was specifically designed and written to support SQLite development. 13The [Fossil repository](https://sqlite.org/src/timeline) contains the urtext. 14 15If you are reading this on GitHub or some other Git repository or service, 16then you are looking at a mirror. The names of check-ins and 17other artifacts in a Git mirror are different from the official 18names for those objects. The offical names for check-ins are 19found in a footer on the check-in comment for authorized mirrors. 20The official check-in name can also be seen in the `manifest.uuid` file 21in the root of the tree. Always use the official name, not the 22Git-name, when communicating about an SQLite check-in. 23 24If you pulled your SQLite source code from a secondary source and want to 25verify its integrity, there are hints on how to do that in the 26[Verifying Code Authenticity](#vauth) section below. 27 28## Obtaining The Code 29 30If you do not want to use Fossil, you can download tarballs or ZIP 31archives or [SQLite archives](https://sqlite.org/cli.html#sqlar) as follows: 32 33 * Lastest trunk check-in as 34 [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz), 35 [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip), or 36 [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar). 37 38 * Latest release as 39 [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release), 40 [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip?r=release), or 41 [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar?r=release). 42 43 * For other check-ins, substitute an appropriate branch name or 44 tag or hash prefix for "release" in the URLs of the previous 45 bullet. Or browse the [timeline](https://www.sqlite.org/src/timeline) 46 to locate the check-in desired, click on its information page link, 47 then click on the "Tarball" or "ZIP Archive" links on the information 48 page. 49 50If you do want to use Fossil to check out the source tree, 51first install Fossil version 2.0 or later. 52(Source tarballs and precompiled binaries available 53[here](https://www.fossil-scm.org/fossil/uv/download.html). Fossil is 54a stand-alone program. To install, simply download or build the single 55executable file and put that file someplace on your $PATH.) 56Then run commands like this: 57 58 mkdir ~/sqlite 59 cd ~/sqlite 60 fossil clone https://www.sqlite.org/src sqlite.fossil 61 fossil open sqlite.fossil 62 63After setting up a repository using the steps above, you can always 64update to the lastest version using: 65 66 fossil update trunk ;# latest trunk check-in 67 fossil update release ;# latest official release 68 69Or type "fossil ui" to get a web-based user interface. 70 71## Compiling 72 73First create a directory in which to place 74the build products. It is recommended, but not required, that the 75build directory be separate from the source directory. Cd into the 76build directory and then from the build directory run the configure 77script found at the root of the source tree. Then run "make". 78 79For example: 80 81 tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite" 82 mkdir bld ;# Build will occur in a sibling directory 83 cd bld ;# Change to the build directory 84 ../sqlite/configure ;# Run the configure script 85 make ;# Run the makefile. 86 make sqlite3.c ;# Build the "amalgamation" source file 87 make test ;# Run some tests (requires Tcl) 88 89See the makefile for additional targets. 90 91The configure script uses autoconf 2.61 and libtool. If the configure 92script does not work out for you, there is a generic makefile named 93"Makefile.linux-gcc" in the top directory of the source tree that you 94can copy and edit to suit your needs. Comments on the generic makefile 95show what changes are needed. 96 97## Using MSVC 98 99On Windows, all applicable build products can be compiled with MSVC. 100First open the command prompt window associated with the desired compiler 101version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE 102with the provided "Makefile.msc" to build one of the supported targets. 103 104For example: 105 106 mkdir bld 107 cd bld 108 nmake /f Makefile.msc TOP=..\sqlite 109 nmake /f Makefile.msc sqlite3.c TOP=..\sqlite 110 nmake /f Makefile.msc sqlite3.dll TOP=..\sqlite 111 nmake /f Makefile.msc sqlite3.exe TOP=..\sqlite 112 nmake /f Makefile.msc test TOP=..\sqlite 113 114There are several build options that can be set via the NMAKE command 115line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument 116to the "sqlite3.dll" command line above. When debugging into the SQLite 117code, adding the "DEBUG=1" argument to one of the above command lines is 118recommended. 119 120SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation 121is required by the makefiles (including those for MSVC). SQLite contains 122a lot of generated code and Tcl is used to do much of that code generation. 123 124## Source Code Tour 125 126Most of the core source files are in the **src/** subdirectory. The 127**src/** folder also contains files used to build the "testfixture" test 128harness. The names of the source files used by "testfixture" all begin 129with "test". 130The **src/** also contains the "shell.c" file 131which is the main program for the "sqlite3.exe" 132[command-line shell](https://sqlite.org/cli.html) and 133the "tclsqlite.c" file which implements the 134[Tcl bindings](https://sqlite.org/tclsqlite.html) for SQLite. 135(Historical note: SQLite began as a Tcl 136extension and only later escaped to the wild as an independent library.) 137 138Test scripts and programs are found in the **test/** subdirectory. 139Addtional test code is found in other source repositories. 140See [How SQLite Is Tested](http://www.sqlite.org/testing.html) for 141additional information. 142 143The **ext/** subdirectory contains code for extensions. The 144Full-text search engine is in **ext/fts3**. The R-Tree engine is in 145**ext/rtree**. The **ext/misc** subdirectory contains a number of 146smaller, single-file extensions, such as a REGEXP operator. 147 148The **tool/** subdirectory contains various scripts and programs used 149for building generated source code files or for testing or for generating 150accessory programs such as "sqlite3_analyzer(.exe)". 151 152### Generated Source Code Files 153 154Several of the C-language source files used by SQLite are generated from 155other sources rather than being typed in manually by a programmer. This 156section will summarize those automatically-generated files. To create all 157of the automatically-generated files, simply run "make target_source". 158The "target_source" make target will create a subdirectory "tsrc/" and 159fill it with all the source files needed to build SQLite, both 160manually-edited files and automatically-generated files. 161 162The SQLite interface is defined by the **sqlite3.h** header file, which is 163generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The 164[Tcl script](http://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion. 165The manifest.uuid file contains the SHA3 hash of the particular check-in 166and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file 167contains the current SQLite version number. The sqlite3.h header is really 168just a copy of src/sqlite.h.in with the source-id and version number inserted 169at just the right spots. Note that comment text in the sqlite3.h file is 170used to generate much of the SQLite API documentation. The Tcl scripts 171used to generate that documentation are in a separate source repository. 172 173The SQL language parser is **parse.c** which is generate from a grammar in 174the src/parse.y file. The conversion of "parse.y" into "parse.c" is done 175by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code 176for lemon is at tool/lemon.c. Lemon uses the tool/lempar.c file as a 177template for generating its parser. 178 179Lemon also generates the **parse.h** header file, at the same time it 180generates parse.c. But the parse.h header file is 181modified further (to add additional symbols) using the ./addopcodes.tcl 182Tcl script. 183 184The **opcodes.h** header file contains macros that define the numbers 185corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h 186file is generated by the scanning the src/vdbe.c source file. The 187Tcl script at ./mkopcodeh.tcl does this scan and generates opcodes.h. 188A second Tcl script, ./mkopcodec.tcl, then scans opcodes.h to generate 189the **opcodes.c** source file, which contains a reverse mapping from 190opcode-number to opcode-name that is used for EXPLAIN output. 191 192The **keywordhash.h** header file contains the definition of a hash table 193that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into 194the numeric codes used by the parse.c parser. The keywordhash.h file is 195generated by a C-language program at tool mkkeywordhash.c. 196 197The **pragma.h** header file contains various definitions used to parse 198and implement the PRAGMA statements. The header is generated by a 199script **tool/mkpragmatab.tcl**. If you want to add a new PRAGMA, edit 200the **tool/mkpragmatab.tcl** file to insert the information needed by the 201parser for your new PRAGMA, then run the script to regenerate the 202**pragma.h** header file. 203 204### The Amalgamation 205 206All of the individual C source code and header files (both manually-edited 207and automatically-generated) can be combined into a single big source file 208**sqlite3.c** called "the amalgamation". The amalgamation is the recommended 209way of using SQLite in a larger application. Combining all individual 210source code files into a single big source code file allows the C compiler 211to perform more cross-procedure analysis and generate better code. SQLite 212runs about 5% faster when compiled from the amalgamation versus when compiled 213from individual source files. 214 215The amalgamation is generated from the tool/mksqlite3c.tcl Tcl script. 216First, all of the individual source files must be gathered into the tsrc/ 217subdirectory (using the equivalent of "make target_source") then the 218tool/mksqlite3c.tcl script is run to copy them all together in just the 219right order while resolving internal "#include" references. 220 221The amalgamation source file is more than 200K lines long. Some symbolic 222debuggers (most notably MSVC) are unable to deal with files longer than 64K 223lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl, 224can be run on the amalgamation to break it up into a single small C file 225called **sqlite3-all.c** that does #include on about seven other files 226named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-7.c**. In this way, 227all of the source code is contained within a single translation unit so 228that the compiler can do extra cross-procedure optimization, but no 229individual source file exceeds 32K lines in length. 230 231## How It All Fits Together 232 233SQLite is modular in design. 234See the [architectural description](http://www.sqlite.org/arch.html) 235for details. Other documents that are useful in 236(helping to understand how SQLite works include the 237[file format](http://www.sqlite.org/fileformat2.html) description, 238the [virtual machine](http://www.sqlite.org/opcode.html) that runs 239prepared statements, the description of 240[how transactions work](http://www.sqlite.org/atomiccommit.html), and 241the [overview of the query planner](http://www.sqlite.org/optoverview.html). 242 243Years of effort have gone into optimizating SQLite, both 244for small size and high performance. And optimizations tend to result in 245complex code. So there is a lot of complexity in the current SQLite 246implementation. It will not be the easiest library in the world to hack. 247 248Key files: 249 250 * **sqlite.h.in** - This file defines the public interface to the SQLite 251 library. Readers will need to be familiar with this interface before 252 trying to understand how the library works internally. 253 254 * **sqliteInt.h** - this header file defines many of the data objects 255 used internally by SQLite. In addition to "sqliteInt.h", some 256 subsystems have their own header files. 257 258 * **parse.y** - This file describes the LALR(1) grammar that SQLite uses 259 to parse SQL statements, and the actions that are taken at each step 260 in the parsing process. 261 262 * **vdbe.c** - This file implements the virtual machine that runs 263 prepared statements. There are various helper files whose names 264 begin with "vdbe". The VDBE has access to the vdbeInt.h header file 265 which defines internal data objects. The rest of SQLite interacts 266 with the VDBE through an interface defined by vdbe.h. 267 268 * **where.c** - This file (together with its helper files named 269 by "where*.c") analyzes the WHERE clause and generates 270 virtual machine code to run queries efficiently. This file is 271 sometimes called the "query optimizer". It has its own private 272 header file, whereInt.h, that defines data objects used internally. 273 274 * **btree.c** - This file contains the implementation of the B-Tree 275 storage engine used by SQLite. The interface to the rest of the system 276 is defined by "btree.h". The "btreeInt.h" header defines objects 277 used internally by btree.c and not published to the rest of the system. 278 279 * **pager.c** - This file contains the "pager" implementation, the 280 module that implements transactions. The "pager.h" header file 281 defines the interface between pager.c and the rest of the system. 282 283 * **os_unix.c** and **os_win.c** - These two files implement the interface 284 between SQLite and the underlying operating system using the run-time 285 pluggable VFS interface. 286 287 * **shell.c.in** - This file is not part of the core SQLite library. This 288 is the file that, when linked against sqlite3.a, generates the 289 "sqlite3.exe" command-line shell. The "shell.c.in" file is transformed 290 into "shell.c" as part of the build process. 291 292 * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It 293 is not part of the core SQLite library. But as most of the tests in this 294 repository are written in Tcl, the Tcl language bindings are important. 295 296 * **test*.c** - Files in the src/ folder that begin with "test" go into 297 building the "testfixture.exe" program. The testfixture.exe program is 298 an enhanced Tcl shell. The testfixture.exe program runs scripts in the 299 test/ folder to validate the core SQLite code. The testfixture program 300 (and some other test programs too) is build and run when you type 301 "make test". 302 303 * **ext/misc/json1.c** - This file implements the various JSON functions 304 that are build into SQLite. 305 306There are many other source files. Each has a succinct header comment that 307describes its purpose and role within the larger system. 308 309<a name="vauth"></a> 310## Verifying Code Authenticity 311 312If you obtained an SQLite source tree from a secondary source, such as a 313GitHub mirror, and you want to verify that it has not been altered, there 314are a couple of ways to do that. 315 316If you have an official release version of SQLite, and you are using the 317`sqlite3.c` amalgamation, then SHA3-256 hashes for the amalgamation are 318available in the [change log](https://www.sqlite.org/changes.html) on 319the official website. After building the `sqlite3.c` file, you can check 320that is authentic by comparing the hash. This does not ensure that the 321test scripts are unaltered, but it does validate the deliverable part of 322the code and only involves computing and comparing a single hash. 323 324For versions other than an official release, or if you are building the 325`sqlite3.c` amalgamation using non-standard build options, the verification 326process is a little more involved. The `manifest` file at the root directory 327of the source tree ([example](https://sqlite.org/src/artifact/bd49a8271d650fa8)) 328contains either a SHA3-256 hash (for newer files) or a SHA1 hash (for 329older files) for every source file in the repository. You can write a script 330to extracts hashes from `manifest` and verifies the hashes against the 331corresponding files in the source tree. The SHA3-256 hash of the `manifest` 332file itself is the official name of the version of the source tree that you 333have. The `manifest.uuid` file should contain the SHA3-256 hash of the 334`manifest` file. If all of the above hash comparisons are correct, then 335you can be confident that your source tree is authentic and unadulterated. 336 337## Contacts 338 339The main SQLite webpage is [http://www.sqlite.org/](http://www.sqlite.org/) 340with geographically distributed backups at 341[http://www2.sqlite.org/](http://www2.sqlite.org) and 342[http://www3.sqlite.org/](http://www3.sqlite.org). 343