1<h1 align="center">SQLite Source Repository</h1> 2 3This repository contains the complete source code for the SQLite database 4engine. Some test scripts are also include. However, many other test scripts 5and most of the documentation are managed separately. 6 7If you are reading this on a Git mirror someplace, you are doing it wrong. 8The [official repository](https://www.sqlite.org/src/) is better. Go there 9now. 10 11## Obtaining The Code 12 13SQLite sources are managed using the 14[Fossil](https://www.fossil-scm.org/), a distributed version control system 15that was specifically designed to support SQLite development. 16If you do not want to use Fossil, you can download tarballs or ZIP 17archives as follows: 18 19 * Lastest trunk check-in: 20 <https://www.sqlite.org/src/tarball/sqlite.tar.gz> or 21 <https://www.sqlite.org/src/zip/sqlite.zip>. 22 23 * Latest release: 24 <https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release> or 25 <https://www.sqlite.org/src/zip/sqlite.zip?r=release>. 26 27 * For other check-ins, substitute an appropriate branch name or 28 tag or hash prefix for "release" in the URLs of the previous 29 bullet. Or browse the [timeline](https://www.sqlite.org/src/timeline) 30 to locate the check-in desired, click on its information page link, 31 then click on the "Tarball" or "ZIP Archive" links on the information 32 page. 33 34If you do want to use Fossil to check out the source tree, 35first install Fossil version 2.0 or later. 36(Source tarballs and precompiled binaries available 37[here](https://www.fossil-scm.org/fossil/uv/download.html). Fossil is 38a stand-alone program. To install, simply download or build the single 39executable file and put that file someplace on your $PATH.) 40Then run commands like this: 41 42 mkdir ~/sqlite 43 cd ~/sqlite 44 fossil clone https://www.sqlite.org/src sqlite.fossil 45 fossil open sqlite.fossil 46 47After setting up a repository using the steps above, you can always 48update to the lastest version using: 49 50 fossil update trunk ;# latest trunk check-in 51 fossil update release ;# latest official release 52 53Or type "fossil ui" to get a web-based user interface. 54 55## Compiling 56 57First create a directory in which to place 58the build products. It is recommended, but not required, that the 59build directory be separate from the source directory. Cd into the 60build directory and then from the build directory run the configure 61script found at the root of the source tree. Then run "make". 62 63For example: 64 65 tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite" 66 mkdir bld ;# Build will occur in a sibling directory 67 cd bld ;# Change to the build directory 68 ../sqlite/configure ;# Run the configure script 69 make ;# Run the makefile. 70 make sqlite3.c ;# Build the "amalgamation" source file 71 make test ;# Run some tests (requires Tcl) 72 73See the makefile for additional targets. 74 75The configure script uses autoconf 2.61 and libtool. If the configure 76script does not work out for you, there is a generic makefile named 77"Makefile.linux-gcc" in the top directory of the source tree that you 78can copy and edit to suit your needs. Comments on the generic makefile 79show what changes are needed. 80 81## Using MSVC 82 83On Windows, all applicable build products can be compiled with MSVC. 84First open the command prompt window associated with the desired compiler 85version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE 86with the provided "Makefile.msc" to build one of the supported targets. 87 88For example: 89 90 mkdir bld 91 cd bld 92 nmake /f Makefile.msc TOP=..\sqlite 93 nmake /f Makefile.msc sqlite3.c TOP=..\sqlite 94 nmake /f Makefile.msc sqlite3.dll TOP=..\sqlite 95 nmake /f Makefile.msc sqlite3.exe TOP=..\sqlite 96 nmake /f Makefile.msc test TOP=..\sqlite 97 98There are several build options that can be set via the NMAKE command 99line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument 100to the "sqlite3.dll" command line above. When debugging into the SQLite 101code, adding the "DEBUG=1" argument to one of the above command lines is 102recommended. 103 104SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation 105is required by the makefiles (including those for MSVC). SQLite contains 106a lot of generated code and Tcl is used to do much of that code generation. 107The makefiles also require AWK. 108 109## Source Code Tour 110 111Most of the core source files are in the **src/** subdirectory. The 112**src/** folder also contains files used to build the "testfixture" test 113harness. The names of the source files used "by testfixture" all begin 114with "test". 115The **src/** also contains the "shell.c" file 116which is the main program for the "sqlite3.exe" 117[command-line shell](https://sqlite.org/cli.html) and 118the "tclsqlite.c" file which implements the 119[TCL bindings](https://sqlite.org/tclsqlite.html) for SQLite. 120(Historical note: SQLite began as a Tcl 121extension and only later escaped to the wild as an independent library.) 122 123Test scripts and programs are found in the **test/** subdirectory. 124Addtional test code is found in other source repositories. 125See [How SQLite Is Tested](http://www.sqlite.org/testing.html) for 126additional information. 127 128The **ext/** subdirectory contains code for extensions. The 129Full-text search engine is in **ext/fts3**. The R-Tree engine is in 130**ext/rtree**. The **ext/misc** subdirectory contains a number of 131smaller, single-file extensions, such as a REGEXP operator. 132 133The **tool/** subdirectory contains various scripts and programs used 134for building generated source code files or for testing or for generating 135accessory programs such as "sqlite3_analyzer(.exe)". 136 137### Generated Source Code Files 138 139Several of the C-language source files used by SQLite are generated from 140other sources rather than being typed in manually by a programmer. This 141section will summarize those automatically-generated files. To create all 142of the automatically-generated files, simply run "make target_source". 143The "target_source" make target will create a subdirectory "tsrc/" and 144fill it with all the source files needed to build SQLite, both 145manually-edited files and automatically-generated files. 146 147The SQLite interface is defined by the **sqlite3.h** header file, which is 148generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The 149[Tcl script](http://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion. 150The manifest.uuid file contains the SHA3 hash of the particular check-in 151and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file 152contains the current SQLite version number. The sqlite3.h header is really 153just a copy of src/sqlite.h.in with the source-id and version number inserted 154at just the right spots. Note that comment text in the sqlite3.h file is 155used to generate much of the SQLite API documentation. The Tcl scripts 156used to generate that documentation are in a separate source repository. 157 158The SQL language parser is **parse.c** which is generate from a grammar in 159the src/parse.y file. The conversion of "parse.y" into "parse.c" is done 160by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code 161for lemon is at tool/lemon.c. Lemon uses a 162template for generating its parser. A generic template is in tool/lempar.c, 163but SQLite uses a slightly modified template found in src/lempar.c. 164 165Lemon also generates the **parse.h** header file, at the same time it 166generates parse.c. But the parse.h header file is 167modified further (to add additional symbols) using the ./addopcodes.awk 168AWK script. 169 170The **opcodes.h** header file contains macros that define the numbers 171corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h 172file is generated by the scanning the src/vdbe.c source file. The 173AWK script at ./mkopcodeh.awk does this scan and generates opcodes.h. 174A second AWK script, ./mkopcodec.awk, then scans opcodes.h to generate 175the **opcodes.c** source file, which contains a reverse mapping from 176opcode-number to opcode-name that is used for EXPLAIN output. 177 178The **keywordhash.h** header file contains the definition of a hash table 179that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into 180the numeric codes used by the parse.c parser. The keywordhash.h file is 181generated by a C-language program at tool mkkeywordhash.c. 182 183Th3 **pragma.h** header file contains various definitions used to parse 184and implement the PRAGMA statements. The header is generated by a 185script **tool/mkpragmatab.tcl**. If you want to add a new PRAGMA, edit 186the **tool/mkpragmatab.tcl** file to insert the information needed by the 187parser for your new PRAGMA, then run the script to regenerate the 188**pragma.h** header file. 189 190### The Amalgamation 191 192All of the individual C source code and header files (both manually-edited 193and automatically-generated) can be combined into a single big source file 194**sqlite3.c** called "the amalgamation". The amalgamation is the recommended 195way of using SQLite in a larger application. Combining all individual 196source code files into a single big source code file allows the C compiler 197to perform more cross-procedure analysis and generate better code. SQLite 198runs about 5% faster when compiled from the amalgamation versus when compiled 199from individual source files. 200 201The amalgamation is generated from the tool/mksqlite3c.tcl Tcl script. 202First, all of the individual source files must be gathered into the tsrc/ 203subdirectory (using the equivalent of "make target_source") then the 204tool/mksqlite3c.tcl script is run to copy them all together in just the 205right order while resolving internal "#include" references. 206 207The amalgamation source file is more than 200K lines long. Some symbolic 208debuggers (most notably MSVC) are unable to deal with files longer than 64K 209lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl, 210can be run on the amalgamation to break it up into a single small C file 211called **sqlite3-all.c** that does #include on about five other files 212named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-5.c**. In this way, 213all of the source code is contained within a single translation unit so 214that the compiler can do extra cross-procedure optimization, but no 215individual source file exceeds 32K lines in length. 216 217## How It All Fits Together 218 219SQLite is modular in design. 220See the [architectural description](http://www.sqlite.org/arch.html) 221for details. Other documents that are useful in 222(helping to understand how SQLite works include the 223[file format](http://www.sqlite.org/fileformat2.html) description, 224the [virtual machine](http://www.sqlite.org/opcode.html) that runs 225prepared statements, the description of 226[how transactions work](http://www.sqlite.org/atomiccommit.html), and 227the [overview of the query planner](http://www.sqlite.org/optoverview.html). 228 229Years of effort have gone into optimizating SQLite, both 230for small size and high performance. And optimizations tend to result in 231complex code. So there is a lot of complexity in the current SQLite 232implementation. It will not be the easiest library in the world to hack. 233 234Key files: 235 236 * **sqlite.h.in** - This file defines the public interface to the SQLite 237 library. Readers will need to be familiar with this interface before 238 trying to understand how the library works internally. 239 240 * **sqliteInt.h** - this header file defines many of the data objects 241 used internally by SQLite. 242 243 * **parse.y** - This file describes the LALR(1) grammer that SQLite uses 244 to parse SQL statements, and the actions that are taken at each step 245 in the parsing process. 246 247 * **vdbe.c** - This file implements the virtual machine that runs 248 prepared statements. There are various helper files whose names 249 begin with "vdbe". The VDBE has access to the vdbeInt.h header file 250 which defines internal data objects. The rest of SQLite interacts 251 with the VDBE through an interface defined by vdbe.h. 252 253 * **where.c** - This file analyzes the WHERE clause and generates 254 virtual machine code to run queries efficiently. This file is 255 sometimes called the "query optimizer". It has its own private 256 header file, whereInt.h, that defines data objects used internally. 257 258 * **btree.c** - This file contains the implementation of the B-Tree 259 storage engine used by SQLite. 260 261 * **pager.c** - This file contains the "pager" implementation, the 262 module that implements transactions. 263 264 * **os_unix.c** and **os_win.c** - These two files implement the interface 265 between SQLite and the underlying operating system using the run-time 266 pluggable VFS interface. 267 268 * **shell.c** - This file is not part of the core SQLite library. This 269 is the file that, when linked against sqlite3.a, generates the 270 "sqlite3.exe" command-line shell. 271 272 * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It 273 is not part of the core SQLite library. But as most of the tests in this 274 repository are written in Tcl, the Tcl language bindings are important. 275 276There are many other source files. Each has a suscinct header comment that 277describes its purpose and role within the larger system. 278 279 280## Contacts 281 282The main SQLite webpage is [http://www.sqlite.org/](http://www.sqlite.org/) 283with geographically distributed backups at 284[http://www2.sqlite.org/](http://www2.sqlite.org) and 285[http://www3.sqlite.org/](http://www3.sqlite.org). 286