1<h1 align="center">SQLite Source Repository</h1> 2 3This repository contains the complete source code for the SQLite database 4engine. Some test scripts are also include. However, many other test scripts 5and most of the documentation are managed separately. 6 7If you are reading this on a Git mirror someplace, you are doing it wrong. 8The [official repository](https://www.sqlite.org/src/) is better. Go there 9now. 10 11## Obtaining The Code 12 13SQLite sources are managed using the 14[Fossil](https://www.fossil-scm.org/), a distributed version control system 15that was specifically designed to support SQLite development. 16If you do not want to use Fossil, you can download tarballs or ZIP 17archives as follows: 18 19 * Lastest trunk check-in: 20 <https://www.sqlite.org/src/tarball/sqlite.tar.gz> or 21 <https://www.sqlite.org/src/zip/sqlite.zip>. 22 23 * Latest release: 24 <https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release> or 25 <https://www.sqlite.org/src/zip/sqlite.zip?r=release>. 26 27 * For other check-ins, substitute an appropriate branch name or 28 tag or hash prefix for "release" in the URLs of the previous 29 bullet. Or browse the [timeline](https://www.sqlite.org/src/timeline) 30 to locate the check-in desired, click on its information page link, 31 then click on the "Tarball" or "ZIP Archive" links on the information 32 page. 33 34If you do want to use Fossil to check out the source tree, 35first install Fossil version 2.0 or later. 36(Source tarballs and precompiled binaries available 37[here](https://www.fossil-scm.org/fossil/uv/download.html).) 38Then run commands like this: 39 40 mkdir ~/sqlite 41 cd ~/sqlite 42 fossil clone https://www.sqlite.org/src sqlite.fossil 43 fossil open sqlite.fossil 44 45After setting up a repository using the steps above, you can always 46update to the lastest version using: 47 48 fossil update trunk ;# latest trunk check-in 49 fossil update release ;# latest official release 50 51Or type "fossil ui" to get a web-based user interface. 52 53## Compiling 54 55First create a directory in which to place 56the build products. It is recommended, but not required, that the 57build directory be separate from the source directory. Cd into the 58build directory and then from the build directory run the configure 59script found at the root of the source tree. Then run "make". 60 61For example: 62 63 tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite" 64 mkdir bld ;# Build will occur in a sibling directory 65 cd bld ;# Change to the build directory 66 ../sqlite/configure ;# Run the configure script 67 make ;# Run the makefile. 68 make sqlite3.c ;# Build the "amalgamation" source file 69 make test ;# Run some tests (requires Tcl) 70 71See the makefile for additional targets. 72 73The configure script uses autoconf 2.61 and libtool. If the configure 74script does not work out for you, there is a generic makefile named 75"Makefile.linux-gcc" in the top directory of the source tree that you 76can copy and edit to suit your needs. Comments on the generic makefile 77show what changes are needed. 78 79## Using MSVC 80 81On Windows, all applicable build products can be compiled with MSVC. 82First open the command prompt window associated with the desired compiler 83version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE 84with the provided "Makefile.msc" to build one of the supported targets. 85 86For example: 87 88 mkdir bld 89 cd bld 90 nmake /f Makefile.msc TOP=..\sqlite 91 nmake /f Makefile.msc sqlite3.c TOP=..\sqlite 92 nmake /f Makefile.msc sqlite3.dll TOP=..\sqlite 93 nmake /f Makefile.msc sqlite3.exe TOP=..\sqlite 94 nmake /f Makefile.msc test TOP=..\sqlite 95 96There are several build options that can be set via the NMAKE command 97line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument 98to the "sqlite3.dll" command line above. When debugging into the SQLite 99code, adding the "DEBUG=1" argument to one of the above command lines is 100recommended. 101 102SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation 103is required by the makefiles (including those for MSVC). SQLite contains 104a lot of generated code and Tcl is used to do much of that code generation. 105The makefiles also require AWK. 106 107## Source Code Tour 108 109Most of the core source files are in the **src/** subdirectory. But 110src/ also contains files used to build the "testfixture" test harness; 111those file all begin with "test". And src/ contains the "shell.c" file 112which is the main program for the "sqlite3.exe" command-line shell and 113the "tclsqlite.c" file which implements the bindings to SQLite from the 114Tcl programming language. (Historical note: SQLite began as a Tcl 115extension and only later escaped to the wild as an independent library.) 116 117Test scripts and programs are found in the **test/** subdirectory. 118There are other test suites for SQLite (see 119[How SQLite Is Tested](http://www.sqlite.org/testing.html)) 120but those other test suites are 121in separate source repositories. 122 123The **ext/** subdirectory contains code for extensions. The 124Full-text search engine is in **ext/fts3**. The R-Tree engine is in 125**ext/rtree**. The **ext/misc** subdirectory contains a number of 126smaller, single-file extensions, such as a REGEXP operator. 127 128The **tool/** subdirectory contains various scripts and programs used 129for building generated source code files or for testing or for generating 130accessory programs such as "sqlite3_analyzer(.exe)". 131 132### Generated Source Code Files 133 134Several of the C-language source files used by SQLite are generated from 135other sources rather than being typed in manually by a programmer. This 136section will summarize those automatically-generated files. To create all 137of the automatically-generated files, simply run "make target_source". 138The "target_source" make target will create a subdirectory "tsrc/" and 139fill it with all the source files needed to build SQLite, both 140manually-edited files and automatically-generated files. 141 142The SQLite interface is defined by the **sqlite3.h** header file, which is 143generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The 144[Tcl script](http://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion. 145The manifest.uuid file contains the SHA1 hash of the particular check-in 146and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file 147contains the current SQLite version number. The sqlite3.h header is really 148just a copy of src/sqlite.h.in with the source-id and version number inserted 149at just the right spots. Note that comment text in the sqlite3.h file is 150used to generate much of the SQLite API documentation. The Tcl scripts 151used to generate that documentation are in a separate source repository. 152 153The SQL language parser is **parse.c** which is generate from a grammar in 154the src/parse.y file. The conversion of "parse.y" into "parse.c" is done 155by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code 156for lemon is at tool/lemon.c. Lemon uses a 157template for generating its parser. A generic template is in tool/lempar.c, 158but SQLite uses a slightly modified template found in src/lempar.c. 159 160Lemon also generates the **parse.h** header file, at the same time it 161generates parse.c. But the parse.h header file is 162modified further (to add additional symbols) using the ./addopcodes.awk 163AWK script. 164 165The **opcodes.h** header file contains macros that define the numbers 166corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h 167file is generated by the scanning the src/vdbe.c source file. The 168AWK script at ./mkopcodeh.awk does this scan and generates opcodes.h. 169A second AWK script, ./mkopcodec.awk, then scans opcodes.h to generate 170the **opcodes.c** source file, which contains a reverse mapping from 171opcode-number to opcode-name that is used for EXPLAIN output. 172 173The **keywordhash.h** header file contains the definition of a hash table 174that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into 175the numeric codes used by the parse.c parser. The keywordhash.h file is 176generated by a C-language program at tool mkkeywordhash.c. 177 178### The Amalgamation 179 180All of the individual C source code and header files (both manually-edited 181and automatically-generated) can be combined into a single big source file 182**sqlite3.c** called "the amalgamation". The amalgamation is the recommended 183way of using SQLite in a larger application. Combining all individual 184source code files into a single big source code file allows the C compiler 185to perform more cross-procedure analysis and generate better code. SQLite 186runs about 5% faster when compiled from the amalgamation versus when compiled 187from individual source files. 188 189The amalgamation is generated from the tool/mksqlite3c.tcl Tcl script. 190First, all of the individual source files must be gathered into the tsrc/ 191subdirectory (using the equivalent of "make target_source") then the 192tool/mksqlite3c.tcl script is run to copy them all together in just the 193right order while resolving internal "#include" references. 194 195The amalgamation source file is more than 100K lines long. Some symbolic 196debuggers (most notably MSVC) are unable to deal with files longer than 64K 197lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl, 198can be run on the amalgamation to break it up into a single small C file 199called **sqlite3-all.c** that does #include on about five other files 200named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-5.c**. In this way, 201all of the source code is contained within a single translation unit so 202that the compiler can do extra cross-procedure optimization, but no 203individual source file exceeds 32K lines in length. 204 205## How It All Fits Together 206 207SQLite is modular in design. 208See the [architectural description](http://www.sqlite.org/arch.html) 209for details. Other documents that are useful in 210(helping to understand how SQLite works include the 211[file format](http://www.sqlite.org/fileformat2.html) description, 212the [virtual machine](http://www.sqlite.org/vdbe.html) that runs 213prepared statements, the description of 214[how transactions work](http://www.sqlite.org/atomiccommit.html), and 215the [overview of the query planner](http://www.sqlite.org/optoverview.html). 216 217Unfortunately, years of effort have gone into optimizating SQLite, both 218for small size and high performance. And optimizations tend to result in 219complex code. So there is a lot of complexity in the SQLite implementation. 220 221Key files: 222 223 * **sqlite.h.in** - This file defines the public interface to the SQLite 224 library. Readers will need to be familiar with this interface before 225 trying to understand how the library works internally. 226 227 * **sqliteInt.h** - this header file defines many of the data objects 228 used internally by SQLite. 229 230 * **parse.y** - This file describes the LALR(1) grammer that SQLite uses 231 to parse SQL statements, and the actions that are taken at each step 232 in the parsing process. 233 234 * **vdbe.c** - This file implements the virtual machine that runs 235 prepared statements. There are various helper files whose names 236 begin with "vdbe". The VDBE has access to the vdbeInt.h header file 237 which defines internal data objects. The rest of SQLite interacts 238 with the VDBE through an interface defined by vdbe.h. 239 240 * **where.c** - This file analyzes the WHERE clause and generates 241 virtual machine code to run queries efficiently. This file is 242 sometimes called the "query optimizer". It has its own private 243 header file, whereInt.h, that defines data objects used internally. 244 245 * **btree.c** - This file contains the implementation of the B-Tree 246 storage engine used by SQLite. 247 248 * **pager.c** - This file contains the "pager" implementation, the 249 module that implements transactions. 250 251 * **os_unix.c** and **os_win.c** - These two files implement the interface 252 between SQLite and the underlying operating system using the run-time 253 pluggable VFS interface. 254 255 * **shell.c** - This file is not part of the core SQLite library. This 256 is the file that, when linked against sqlite3.a, generates the 257 "sqlite3.exe" command-line shell. 258 259 * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It 260 is not part of the core SQLite library. But as most of the tests in this 261 repository are written in Tcl, the Tcl language bindings are important. 262 263There are many other source files. Each has a suscinct header comment that 264describes its purpose and role within the larger system. 265 266 267## Contacts 268 269The main SQLite webpage is [http://www.sqlite.org/](http://www.sqlite.org/) 270with geographically distributed backup servers at 271[http://www2.sqlite.org/](http://www2.sqlite.org) and 272[http://www3.sqlite.org/](http://www3.sqlite.org). 273