1<h1 align="center">SQLite Source Repository</h1> 2 3This repository contains the complete source code for the SQLite database 4engine. Some test scripts are also include. However, many other test scripts 5and most of the documentation are managed separately. 6 7If you are reading this on a Git mirror someplace, you are doing it wrong. 8The [official repository](https://www.sqlite.org/src/) is better. Go there 9now. 10 11## Compiling 12 13First create a directory in which to place 14the build products. It is recommended, but not required, that the 15build directory be separate from the source directory. Cd into the 16build directory and then from the build directory run the configure 17script found at the root of the source tree. Then run "make". 18 19For example: 20 21 tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite" 22 mkdir bld ;# Build will occur in a sibling directory 23 cd bld ;# Change to the build directory 24 ../sqlite/configure ;# Run the configure script 25 make ;# Run the makefile. 26 make sqlite3.c ;# Build the "amalgamation" source file 27 make test ;# Run some tests (requires Tcl) 28 29See the makefile for additional targets. 30 31The configure script uses autoconf 2.61 and libtool. If the configure 32script does not work out for you, there is a generic makefile named 33"Makefile.linux-gcc" in the top directory of the source tree that you 34can copy and edit to suit your needs. Comments on the generic makefile 35show what changes are needed. 36 37## Using MSVC 38 39On Windows, all applicable build products can be compiled with MSVC. 40First open the command prompt window associated with the desired compiler 41version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE 42with the provided "Makefile.msc" to build one of the supported targets. 43 44For example: 45 46 mkdir bld 47 cd bld 48 nmake /f Makefile.msc TOP=..\sqlite 49 nmake /f Makefile.msc sqlite3.c TOP=..\sqlite 50 nmake /f Makefile.msc sqlite3.dll TOP=..\sqlite 51 nmake /f Makefile.msc sqlite3.exe TOP=..\sqlite 52 nmake /f Makefile.msc test TOP=..\sqlite 53 54There are several build options that can be set via the NMAKE command 55line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument 56to the "sqlite3.dll" command line above. When debugging into the SQLite 57code, adding the "DEBUG=1" argument to one of the above command lines is 58recommended. 59 60SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation 61is required by the makefiles (including those for MSVC). SQLite contains 62a lot of generated code and Tcl is used to do much of that code generation. 63The makefiles also require AWK. 64 65## Source Code Tour 66 67Most of the core source files are in the **src/** subdirectory. But 68src/ also contains files used to build the "testfixture" test harness; 69those file all begin with "test". And src/ contains the "shell.c" file 70which is the main program for the "sqlite3.exe" command-line shell and 71the "tclsqlite.c" file which implements the bindings to SQLite from the 72Tcl programming language. (Historical note: SQLite began as a Tcl 73extension and only later escaped to the wild as an independent library.) 74 75Test scripts and programs are found in the **test/** subdirectory. 76There are other test suites for SQLite (see 77[How SQLite Is Tested](http://www.sqlite.org/testing.html)) 78but those other test suites are 79in separate source repositories. 80 81The **ext/** subdirectory contains code for extensions. The 82Full-text search engine is in **ext/fts3**. The R-Tree engine is in 83**ext/rtree**. The **ext/misc** subdirectory contains a number of 84smaller, single-file extensions, such as a REGEXP operator. 85 86The **tool/** subdirectory contains various scripts and programs used 87for building generated source code files or for testing or for generating 88accessory programs such as "sqlite3_analyzer(.exe)". 89 90### Generated Source Code Files 91 92Several of the C-language source files used by SQLite are generated from 93other sources rather than being typed in manually by a programmer. This 94section will summarize those automatically-generated files. To create all 95of the automatically-generated files, simply run "make target_source". 96The "target_source" make target will create a subdirectory "tsrc/" and 97fill it with all the source files needed to build SQLite, both 98manually-edited files and automatically-generated files. 99 100The SQLite interface is defined by the **sqlite3.h** header file, which is 101generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The 102[Tcl script](http://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion. 103The manifest.uuid file contains the SHA1 hash of the particular check-in 104and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file 105contains the current SQLite version number. The sqlite3.h header is really 106just a copy of src/sqlite.h.in with the source-id and version number inserted 107at just the right spots. Note that comment text in the sqlite3.h file is 108used to generate much of the SQLite API documentation. The Tcl scripts 109used to generate that documentation are in a separate source repository. 110 111The SQL language parser is **parse.c** which is generate from a grammar in 112the src/parse.y file. The conversion of "parse.y" into "parse.c" is done 113by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code 114for lemon is at tool/lemon.c. Lemon uses a 115template for generating its parser. A generic template is in tool/lempar.c, 116but SQLite uses a slightly modified template found in src/lempar.c. 117 118Lemon also generates the **parse.h** header file, at the same time it 119generates parse.c. But the parse.h header file is 120modified further (to add additional symbols) using the ./addopcodes.awk 121AWK script. 122 123The **opcodes.h** header file contains macros that define the numbers 124corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h 125file is generated by the scanning the src/vdbe.c source file. The 126AWK script at ./mkopcodeh.awk does this scan and generates opcodes.h. 127A second AWK script, ./mkopcodec.awk, then scans opcodes.h to generate 128the **opcodes.c** source file, which contains a reverse mapping from 129opcode-number to opcode-name that is used for EXPLAIN output. 130 131The **keywordhash.h** header file contains the definition of a hash table 132that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into 133the numeric codes used by the parse.c parser. The keywordhash.h file is 134generated by a C-language program at tool mkkeywordhash.c. 135 136### The Amalgamation 137 138All of the individual C source code and header files (both manually-edited 139and automatically-generated) can be combined into a single big source file 140**sqlite3.c** called "the amalgamation". The amalgamation is the recommended 141way of using SQLite in a larger application. Combining all individual 142source code files into a single big source code file allows the C compiler 143to perform more cross-procedure analysis and generate better code. SQLite 144runs about 5% faster when compiled from the amalgamation versus when compiled 145from individual source files. 146 147The amalgamation is generated from the tool/mksqlite3c.tcl Tcl script. 148First, all of the individual source files must be gathered into the tsrc/ 149subdirectory (using the equivalent of "make target_source") then the 150tool/mksqlite3c.tcl script is run to copy them all together in just the 151right order while resolving internal "#include" references. 152 153The amalgamation source file is more than 100K lines long. Some symbolic 154debuggers (most notably MSVC) are unable to deal with files longer than 64K 155lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl, 156can be run on the amalgamation to break it up into a single small C file 157called **sqlite3-all.c** that does #include on about five other files 158named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-5.c**. In this way, 159all of the source code is contained within a single translation unit so 160that the compiler can do extra cross-procedure optimization, but no 161individual source file exceeds 32K lines in length. 162 163## How It All Fits Together 164 165SQLite is modular in design. 166See the [architectural description](http://www.sqlite.org/arch.html) 167for details. Other documents that are useful in 168(helping to understand how SQLite works include the 169[file format](http://www.sqlite.org/fileformat2.html) description, 170the [virtual machine](http://www.sqlite.org/vdbe.html) that runs 171prepared statements, the description of 172[how transactions work](http://www.sqlite.org/atomiccommit.html), and 173the [overview of the query planner](http://www.sqlite.org/optoverview.html). 174 175Unfortunately, years of effort have gone into optimizating SQLite, both 176for small size and high performance. And optimizations tend to result in 177complex code. So there is a lot of complexity in the SQLite implementation. 178 179Key files: 180 181 * **sqlite.h.in** - This file defines the public interface to the SQLite 182 library. Readers will need to be familiar with this interface before 183 trying to understand how the library works internally. 184 185 * **sqliteInt.h** - this header file defines many of the data objects 186 used internally by SQLite. 187 188 * **parse.y** - This file describes the LALR(1) grammer that SQLite uses 189 to parse SQL statements, and the actions that are taken at each step 190 in the parsing process. 191 192 * **vdbe.c** - This file implements the virtual machine that runs 193 prepared statements. There are various helper files whose names 194 begin with "vdbe". The VDBE has access to the vdbeInt.h header file 195 which defines internal data objects. The rest of SQLite interacts 196 with the VDBE through an interface defined by vdbe.h. 197 198 * **where.c** - This file analyzes the WHERE clause and generates 199 virtual machine code to run queries efficiently. This file is 200 sometimes called the "query optimizer". It has its own private 201 header file, whereInt.h, that defines data objects used internally. 202 203 * **btree.c** - This file contains the implementation of the B-Tree 204 storage engine used by SQLite. 205 206 * **pager.c** - This file contains the "pager" implementation, the 207 module that implements transactions. 208 209 * **os_unix.c** and **os_win.c** - These two files implement the interface 210 between SQLite and the underlying operating system using the run-time 211 pluggable VFS interface. 212 213 * **shell.c** - This file is not part of the core SQLite library. This 214 is the file that, when linked against sqlite3.a, generates the 215 "sqlite3.exe" command-line shell. 216 217 * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It 218 is not part of the core SQLite library. But as most of the tests in this 219 repository are written in Tcl, the Tcl language bindings are important. 220 221There are many other source files. Each has a suscinct header comment that 222describes its purpose and role within the larger system. 223 224 225## Contacts 226 227The main SQLite webpage is [http://www.sqlite.org/](http://www.sqlite.org/) 228with geographically distributed backup servers at 229[http://www2.sqlite.org/](http://www2.sqlite.org) and 230[http://www3.sqlite.org/](http://www3.sqlite.org). 231