1<h1 align="center">SQLite Source Repository</h1> 2 3This repository contains the complete source code for the SQLite database 4engine. Some test scripts are also include. However, many other test scripts 5and most of the documentation are managed separately. 6 7If you are reading this on a Git mirror someplace, you are doing it wrong. 8The [official repository](https://www.sqlite.org/src/) is better. Go there 9now. 10 11## Obtaining The Code 12 13SQLite sources are managed using the 14[Fossil](https://www.fossil-scm.org/), a distributed version control system 15that was specifically designed to support SQLite development. 16If you do not want to use Fossil, you can download tarballs or ZIP 17archives as follows: 18 19 * Lastest trunk check-in: 20 <https://www.sqlite.org/src/tarball/sqlite.tar.gz> or 21 <https://www.sqlite.org/src/zip/sqlite.zip>. 22 23 * Latest release: 24 <https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release> or 25 <https://www.sqlite.org/src/zip/sqlite.zip?r=release>. 26 27 * For other check-ins, substitute an appropriate branch name or 28 tag or hash prefix for "release" in the URLs of the previous 29 bullet. Or browse the [timeline](https://www.sqlite.org/src/timeline) 30 to locate the check-in desired, click on its information page link, 31 then click on the "Tarball" or "ZIP Archive" links on the information 32 page. 33 34If you do want to use Fossil to check out the source tree, 35first install Fossil version 2.0 or later. 36(Source tarballs and precompiled binaries available 37[here](https://www.fossil-scm.org/fossil/uv/download.html). Fossil is 38a stand-alone program. To install, simply download or build the single 39executable file and put that file someplace on your $PATH.) 40Then run commands like this: 41 42 mkdir ~/sqlite 43 cd ~/sqlite 44 fossil clone https://www.sqlite.org/src sqlite.fossil 45 fossil open sqlite.fossil 46 47After setting up a repository using the steps above, you can always 48update to the lastest version using: 49 50 fossil update trunk ;# latest trunk check-in 51 fossil update release ;# latest official release 52 53Or type "fossil ui" to get a web-based user interface. 54 55## Compiling 56 57First create a directory in which to place 58the build products. It is recommended, but not required, that the 59build directory be separate from the source directory. Cd into the 60build directory and then from the build directory run the configure 61script found at the root of the source tree. Then run "make". 62 63For example: 64 65 tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite" 66 mkdir bld ;# Build will occur in a sibling directory 67 cd bld ;# Change to the build directory 68 ../sqlite/configure ;# Run the configure script 69 make ;# Run the makefile. 70 make sqlite3.c ;# Build the "amalgamation" source file 71 make test ;# Run some tests (requires Tcl) 72 73See the makefile for additional targets. 74 75The configure script uses autoconf 2.61 and libtool. If the configure 76script does not work out for you, there is a generic makefile named 77"Makefile.linux-gcc" in the top directory of the source tree that you 78can copy and edit to suit your needs. Comments on the generic makefile 79show what changes are needed. 80 81## Using MSVC 82 83On Windows, all applicable build products can be compiled with MSVC. 84First open the command prompt window associated with the desired compiler 85version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE 86with the provided "Makefile.msc" to build one of the supported targets. 87 88For example: 89 90 mkdir bld 91 cd bld 92 nmake /f Makefile.msc TOP=..\sqlite 93 nmake /f Makefile.msc sqlite3.c TOP=..\sqlite 94 nmake /f Makefile.msc sqlite3.dll TOP=..\sqlite 95 nmake /f Makefile.msc sqlite3.exe TOP=..\sqlite 96 nmake /f Makefile.msc test TOP=..\sqlite 97 98There are several build options that can be set via the NMAKE command 99line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument 100to the "sqlite3.dll" command line above. When debugging into the SQLite 101code, adding the "DEBUG=1" argument to one of the above command lines is 102recommended. 103 104SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation 105is required by the makefiles (including those for MSVC). SQLite contains 106a lot of generated code and Tcl is used to do much of that code generation. 107The makefiles also require AWK. 108 109## Source Code Tour 110 111Most of the core source files are in the **src/** subdirectory. The 112**src/** folder also contains files used to build the "testfixture" test 113harness. The names of the source files used by "testfixture" all begin 114with "test". 115The **src/** also contains the "shell.c" file 116which is the main program for the "sqlite3.exe" 117[command-line shell](https://sqlite.org/cli.html) and 118the "tclsqlite.c" file which implements the 119[TCL bindings](https://sqlite.org/tclsqlite.html) for SQLite. 120(Historical note: SQLite began as a Tcl 121extension and only later escaped to the wild as an independent library.) 122 123Test scripts and programs are found in the **test/** subdirectory. 124Addtional test code is found in other source repositories. 125See [How SQLite Is Tested](http://www.sqlite.org/testing.html) for 126additional information. 127 128The **ext/** subdirectory contains code for extensions. The 129Full-text search engine is in **ext/fts3**. The R-Tree engine is in 130**ext/rtree**. The **ext/misc** subdirectory contains a number of 131smaller, single-file extensions, such as a REGEXP operator. 132 133The **tool/** subdirectory contains various scripts and programs used 134for building generated source code files or for testing or for generating 135accessory programs such as "sqlite3_analyzer(.exe)". 136 137### Generated Source Code Files 138 139Several of the C-language source files used by SQLite are generated from 140other sources rather than being typed in manually by a programmer. This 141section will summarize those automatically-generated files. To create all 142of the automatically-generated files, simply run "make target_source". 143The "target_source" make target will create a subdirectory "tsrc/" and 144fill it with all the source files needed to build SQLite, both 145manually-edited files and automatically-generated files. 146 147The SQLite interface is defined by the **sqlite3.h** header file, which is 148generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The 149[Tcl script](http://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion. 150The manifest.uuid file contains the SHA3 hash of the particular check-in 151and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file 152contains the current SQLite version number. The sqlite3.h header is really 153just a copy of src/sqlite.h.in with the source-id and version number inserted 154at just the right spots. Note that comment text in the sqlite3.h file is 155used to generate much of the SQLite API documentation. The Tcl scripts 156used to generate that documentation are in a separate source repository. 157 158The SQL language parser is **parse.c** which is generate from a grammar in 159the src/parse.y file. The conversion of "parse.y" into "parse.c" is done 160by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code 161for lemon is at tool/lemon.c. Lemon uses the tool/lempar.c file as a 162template for generating its parser. 163 164Lemon also generates the **parse.h** header file, at the same time it 165generates parse.c. But the parse.h header file is 166modified further (to add additional symbols) using the ./addopcodes.awk 167AWK script. 168 169The **opcodes.h** header file contains macros that define the numbers 170corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h 171file is generated by the scanning the src/vdbe.c source file. The 172AWK script at ./mkopcodeh.awk does this scan and generates opcodes.h. 173A second AWK script, ./mkopcodec.awk, then scans opcodes.h to generate 174the **opcodes.c** source file, which contains a reverse mapping from 175opcode-number to opcode-name that is used for EXPLAIN output. 176 177The **keywordhash.h** header file contains the definition of a hash table 178that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into 179the numeric codes used by the parse.c parser. The keywordhash.h file is 180generated by a C-language program at tool mkkeywordhash.c. 181 182The **pragma.h** header file contains various definitions used to parse 183and implement the PRAGMA statements. The header is generated by a 184script **tool/mkpragmatab.tcl**. If you want to add a new PRAGMA, edit 185the **tool/mkpragmatab.tcl** file to insert the information needed by the 186parser for your new PRAGMA, then run the script to regenerate the 187**pragma.h** header file. 188 189### The Amalgamation 190 191All of the individual C source code and header files (both manually-edited 192and automatically-generated) can be combined into a single big source file 193**sqlite3.c** called "the amalgamation". The amalgamation is the recommended 194way of using SQLite in a larger application. Combining all individual 195source code files into a single big source code file allows the C compiler 196to perform more cross-procedure analysis and generate better code. SQLite 197runs about 5% faster when compiled from the amalgamation versus when compiled 198from individual source files. 199 200The amalgamation is generated from the tool/mksqlite3c.tcl Tcl script. 201First, all of the individual source files must be gathered into the tsrc/ 202subdirectory (using the equivalent of "make target_source") then the 203tool/mksqlite3c.tcl script is run to copy them all together in just the 204right order while resolving internal "#include" references. 205 206The amalgamation source file is more than 200K lines long. Some symbolic 207debuggers (most notably MSVC) are unable to deal with files longer than 64K 208lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl, 209can be run on the amalgamation to break it up into a single small C file 210called **sqlite3-all.c** that does #include on about five other files 211named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-5.c**. In this way, 212all of the source code is contained within a single translation unit so 213that the compiler can do extra cross-procedure optimization, but no 214individual source file exceeds 32K lines in length. 215 216## How It All Fits Together 217 218SQLite is modular in design. 219See the [architectural description](http://www.sqlite.org/arch.html) 220for details. Other documents that are useful in 221(helping to understand how SQLite works include the 222[file format](http://www.sqlite.org/fileformat2.html) description, 223the [virtual machine](http://www.sqlite.org/opcode.html) that runs 224prepared statements, the description of 225[how transactions work](http://www.sqlite.org/atomiccommit.html), and 226the [overview of the query planner](http://www.sqlite.org/optoverview.html). 227 228Years of effort have gone into optimizating SQLite, both 229for small size and high performance. And optimizations tend to result in 230complex code. So there is a lot of complexity in the current SQLite 231implementation. It will not be the easiest library in the world to hack. 232 233Key files: 234 235 * **sqlite.h.in** - This file defines the public interface to the SQLite 236 library. Readers will need to be familiar with this interface before 237 trying to understand how the library works internally. 238 239 * **sqliteInt.h** - this header file defines many of the data objects 240 used internally by SQLite. 241 242 * **parse.y** - This file describes the LALR(1) grammar that SQLite uses 243 to parse SQL statements, and the actions that are taken at each step 244 in the parsing process. 245 246 * **vdbe.c** - This file implements the virtual machine that runs 247 prepared statements. There are various helper files whose names 248 begin with "vdbe". The VDBE has access to the vdbeInt.h header file 249 which defines internal data objects. The rest of SQLite interacts 250 with the VDBE through an interface defined by vdbe.h. 251 252 * **where.c** - This file analyzes the WHERE clause and generates 253 virtual machine code to run queries efficiently. This file is 254 sometimes called the "query optimizer". It has its own private 255 header file, whereInt.h, that defines data objects used internally. 256 257 * **btree.c** - This file contains the implementation of the B-Tree 258 storage engine used by SQLite. 259 260 * **pager.c** - This file contains the "pager" implementation, the 261 module that implements transactions. 262 263 * **os_unix.c** and **os_win.c** - These two files implement the interface 264 between SQLite and the underlying operating system using the run-time 265 pluggable VFS interface. 266 267 * **shell.c** - This file is not part of the core SQLite library. This 268 is the file that, when linked against sqlite3.a, generates the 269 "sqlite3.exe" command-line shell. 270 271 * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It 272 is not part of the core SQLite library. But as most of the tests in this 273 repository are written in Tcl, the Tcl language bindings are important. 274 275There are many other source files. Each has a succinct header comment that 276describes its purpose and role within the larger system. 277 278 279## Contacts 280 281The main SQLite webpage is [http://www.sqlite.org/](http://www.sqlite.org/) 282with geographically distributed backups at 283[http://www2.sqlite.org/](http://www2.sqlite.org) and 284[http://www3.sqlite.org/](http://www3.sqlite.org). 285