First public commit

1 files changed, 199 insertions, 0 deletions

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..7c71810
--- /dev/null
+++ b/README.md
@@ -0,0 +1,199 @@
+# **Shoelaces:** lightweight and painless server bootstrapping
+
+**Shoelaces** serves [iPXE](https://ipxe.org/) boot scripts,
+[cloud-init](http://cloud-init.org/) configuration, and any other configuration
+files over HTTP to hardware or virtual machines booting over iPXE. It also does
+a few other things to make it easier to manage your server deployments:
+
+* Has a simple but **nice UI** to show the current configuration, and history of
+  servers that booted.
+* Uses simple **Go based template language** to handle more complex configurations.
+* Allows specifying the **boot entry point** for a given server based on its
+  **IP** address or **DNS PTR** record.
+* Supports the notion of **environments** for _Development_ and _Production_
+  environment configurations, while trying to minimize template duplication.
+* Puts unknown servers into iPXE script boot **retry loop**, while at the same
+  time **showing them in the UI** allowing the user to select a specific boot
+  configuration.
+
+## How it works
+
+![Shoelaces overview](shoelaces-overview.png)
+
+The previous figure shows a high level overview of how a server contacts
+Shoelaces. We can see that as soon as the server boots using network boot, we
+instruct the machine to switch to an [iPXE](https://ipxe.org/) ROM. We do this
+because we need to be able to make HTTP requests to Shoelaces, and regular
+[PXE](https://en.wikipedia.org/wiki/Preboot_Execution_Environment) does not
+support that protocol.
+
+So, when a server boots, the
+[DHCP](https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol) server
+will instruct it to retrieve an iPXE ROM from a
+[TFTP](https://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol) server.
+When the host receives the iPXE ROM, it will chainload into it and trigger a new
+DHCP request. Finally, the server will detect that the request comes from an
+iPXE ROM, allowing it to respond with an HTTP URL. This URL, as you may have
+guessed, will be pointing to Shoelaces.
+
+
+## Setting up
+
+### Building Shoelaces
+At the moment a binary package is not provided. The only way of running
+Shoelaces is to compile it from source. Refer to the Go Programming
+Language [Getting Started](https://golang.org/doc/install) guide to learn
+how to compile Shoelaces.
+
+Once that you have configured your Go, you can get and compile Shoelaces by
+running:
+
+    $ go get github.com/thousandeyes/shoelaces
+    $ cd $GOPATH/src/github.com/thousandeyes/shoelaces
+    $ go build
+
+### Running Shoelaces
+You can quickly try Shoelaces after compiling it by using the example configuration file:
+
+    ./shoelaces -config configs/shoelaces.conf
+
+Head to [localhost:8081](http://localhost:8081) to checkout Shoelaces' frontend.
+
+### Shoelaces configuration file
+Shoelaces accepts several parameters:
+
+* `config`: the path to a configuration file.
+* `data-dir`: the path to the root directory with the templates. It's advised to
+  manage the templates in a VCS, such as a git repository. Refer to the [example
+  data directory](configs/example-templates-configs/) for more information.
+* `debug`: enable debug messages.
+* `domain`: the domain Shoelaces is going to be listening on.
+* `mappings-file`: the path to the YAML mappings file, relative to the `data-dir` parameter.
+* `port`: the port Shoelaces will listen on.
+* `template-extension`: the filename extension for the templates. The default is
+  `.slc`, so you can just stick with that.
+
+The parameters can be specified in a configuration file, as environment
+variables or, of course, as parameters when running the Shoelaces binary.
+
+Refer to the [example config file](configs/shoelaces.conf) for more information.
+
+### Extra requirements
+
+Along with your **Shoelaces** installation, you will need a LAN segment with
+working [TFTP](https://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol) and
+[DHCP](https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol)
+servers. Any TFTP server should work. The DHCP server will need to be able to
+match the `user-class` of the boot client. In our example the configuration is
+for the widely used [ISC DHCP Server](https://www.isc.org/downloads/dhcp/).
+Shoelaces will happily coexist with the TFTP and DHCP servers on the same host.
+The server you are going to bootstrap needs to be capable of booting over the
+network using
+[PXE](https://en.wikipedia.org/wiki/Preboot_Execution_Environment).
+
+#### TFTP
+
+The TFTP server is only used to chainload the iPXE boot loader, so setting it up
+in `read-only` mode is sufficient. The loader we use (`undionly.kpxe`) can be
+downloaded from the [ipxe.org](http://ipxe.org/howto/chainloading) website.
+
+It is also possible to compile your own iPXE ROM in order to customize the
+booting of your servers. For example, it's useful to [add your own SSL
+certificates](http://ipxe.org/crypto#trusted_root_certificates) in case you want
+to boot using HTTPS.
+
+#### DHCP
+
+Drop this config in your **ISC DHCP** server, replacing the relevant sections
+with your TFTP and Shoelaces server addresses.
+
+```
+# dhcp.conf
+next-server <your-tftp-server>;
+if exists user-class and option user-class = "iPXE" {
+  filename "http://<shoelaces-server>/poll/1/${netX/mac:hexhyp}";
+} else {
+  filename "undionly.kpxe";
+}
+```
+
+The **${netX/mac:hexhyp}** strings represents the MAC address of the booting
+host. iPXE will be in charge of replacing that string for the actual value.
+
+*Note*: In case you are using a DHCP server that does not have this level of
+flexibility for configuring it, you can always re-compile the iPXE ROM for
+[breaking the loop](https://ipxe.org/howto/chainloading#breaking_the_loop_with_an_embedded_script).
+
+
+## Script discoverability
+
+The purpose of Shoelaces is automation. The less input it receives from the
+user, the better. When a server boots, Shoelaces needs the user to select the
+booting script to use, but there are certain cases where we can automate even
+that.
+
+* You can preload Shoelaces with mappings from **IPs to boot scripts**.
+* You can preload Shoelaces with mappings from **hostnames to boot scripts**. When a
+  server boots, Shoelaces will make a reverse DNS query to get the hostname for
+  the IP that made the request, and will match the result to a series of regular
+  expressions.
+
+Shoelaces will read these mappings from a YAML file that can be passed as a
+program parameter. Refer to the [example mappings
+file](configs/example-templates-configs/mappings.yaml) for more information.
+
+## Environments
+
+Shoelaces supports the notion of environments a.k.a. *env overrides*.
+Consider the following `data-dir` directory structure:
+```
+├── cloud-config
+│   └── coreos-cloud-config.yaml.slc
+├── env_overrides
+|   └── testing
+|       └─── cloud-config
+|            └── coreos-cloud-config.yaml.slc
+├── ipxe
+│   ├── coreos.ipxe.slc
+│   └── ubuntu-minimal.ipxe.slc
+├── mappings.yaml
+├── preseed
+│   └── common.preseed.slc
+└── static
+    ├── bootstrap.sh
+    └── rc.local-bootstrap
+```
+
+In this case, hosts that have `environment: testing` set in the `mappings.yaml`
+will be assigned the `testing` environment and they'll use the
+`coreos-cloud-config.yaml.slc` template from the `env_overrides/testing
+directory`, while the rest of the templates will be served from the base
+directory. Everything except `mappings.yaml` can be put in `env_overrides/$env`
+preserving the path.
+
+The way this works, considering that **Shoelaces** is mostly stateless, is by
+setting different `baseURL` depending on the environment set. Normal requests
+would get `baseURL` set to `http://$shoelaces_host:$port` while an environment
+request will have `http://$shoelaces_host:$port/env/$environment_name/`
+
+
+*CORNER CASES*: It is not possible to boot a host in a non default environment
+unless there is a main iPXE script in the respective override directory. This
+means /ipxemenu will only present default and non-default **iPXE** entry points,
+and if you have a template that's included later in the boot process as an
+override you won't be able to select it.
+
+## Contributing
+
+Contributions to Shoelaces are very welcome! Take into account the following
+guidelines:
+
+* [File an issue](https://github.com/thousandeyes/shoelaces/issues) if you find
+  a bug or, even better, contribute with a pull request.
+* We have a bunch of integration tests that can be run by executing `make test`.
+  Ensure that all test pass before submitting your pull request.
+* We are using [dep](https://golang.github.io/dep/) for managing the
+  dependencies of this project. If your contribution adds a new dependency (not
+  recommended, but sometimes there isn't other way), please update dep files and
+  vendor directory
+  [accordingly](https://golang.github.io/dep/docs/daily-dep.html#adding-a-new-dependency).