From 73ab23b5f74174ec6b6a988103306eeec389d5a8 Mon Sep 17 00:00:00 2001 From: Mike Crute Date: Tue, 16 Nov 2021 21:50:31 -0800 Subject: Add some docs --- README.md | 205 ++++++++++++++++++++++++++++++++++++++++++++++++- README_BROKER.md | 228 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 431 insertions(+), 2 deletions(-) create mode 100644 README_BROKER.md diff --git a/README.md b/README.md index ca08bbd..1a892a5 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,207 @@ # Cloud Identity Broker -// TODO: https://docs.gitlab.com/ee/api/jobs.html#get-job-tokens-job -// TODO: Admin UI +This is a cloud identity broker. Identity brokers exist to bridge +authentication from one system into authentication to another system. In the +case of this broker it bridges GitHub and GitLab authentication into temporary +credentials for cloud providers. The broker has an interactive API that can be +used in a web browser as well as a JSON/REST API that can be consumed +programmatically. +The reason this broker exists is to provide one secured location for long-lived +cloud provider credentials that can be easily consumed from development +environments and build systems while limiting the scope of access and time frame +of access those systems have to cloud resources. This helps to increase the +security of those accounts and reduce the risk of abuse. All access through the +broker is chained to an authenticated and authorized user account and all +access through those user accounts are logged for auditing. All credentials +returned by the broker have a strict time limit after which they are useless. +This broker started life as a piece of internal infrastructure specific to AWS +that has been re-written and open sourced with the goal to support multiple +clouds. + +For documentation the REST API look at the `README_BROKER.md` file. + +## Contributing + +Contributions are welcomed. Please send contributions to mike-at-crute-dot-us +either through email in `git format-patch` format or send a link to a git repo +and branch name that the author can pull and merge. + +If you're reading this on one of the author's git forge accounts (GitHub, +GitLab, sr.ht, etc...) feel free to open a pull/merge request with your +changes. + +The author reserves the right to request changes to contributions before +merging them or to not merge them at all but will endeavour to be reasonable in +those requests. + +## Building + +This requires Go 1.17 or newer to build. Provided that a simple `make` should +create a `cloud-identity-broker` binary that's ready to use. + +The built binary embeds all of the templates in production mode and will +require nothing more than the binary itself to run, if running in `--debug` +mode then the `templates` folder must be available. + +## Running + +It's complicated, so probably don't? + +The application requires some bits of infrastructure to be in place for it to +run but then can be exposed directly to the internet or put behind a reverse +proxy. You will need: + +- [Mongodb](https://www.mongodb.com/) +- [Vault](https://www.hashicorp.com/products/vault) +- [GitHub Oauth Application](https://docs.github.com/en/developers/apps/building-oauth-apps/creating-an-oauth-app) +- [SSL certificates](https://letsencrypt.org/). + +Once the requisite infrastructure (see below) is configured, run the binary +like so: + +``` +VAULT_ROLE_ID="..." \ +VAULT_SECRET_ID="..." \ +VAULT_ADDR="https://your-vault-addr:8200" \ + ./cloud-identity-broker \ + --mongodb-uri="mongodb://your-mongodb-host:27017/your-db-name?authSource=admin" \ + --mongodb-vault-path="database/static-creds/your-cred-name" \ + --github-oauth-vault-path="service/service-name/github-oauth" \ + web +``` + +### SSL Certificates + +SSL certificates of a cipher type compatible with Go's SSL library are required +to run the binary, even in development mode. Obtain those certificates however +you would normal do so. + +Both a key and certificate are required to be stored in the directory +identified by `--tls-cache-dir` (which defaults to `./ssl/`). Both files should +be of PEM format with the certificate named `cert.pem` and the key named +`key.pem`. + +### Vault Setup + +Configure Vault per the manufacturers instructions. Create an AppRole for the +identity broker and provide the credentials to the binary using the standard +Vault environment variables. + +The binary will need access to a set of GitHub Oauth client credentials for a +GitHub Oauth application that is used to authenticate users to the broker. +Setup an application on GitHub and store the client ID and client secret in a +JSON document in Vault the vault KV backend with the following format: + +``` +{ + "client-id": "your client id", + "client-secret": "your client secret" +} +``` + +Provide the path to this material with the `--github-oauth-vault-path` command +line argument. You should omit the `kv/data/` prefix to the path. + +For each AWS cloud account to which the broker provides access a JSON document +must be stored in the Vault KV backend. The format of this document is as +follows. The path to this document is provided (with the `kv/data/` prefix +omitted) in the `VaultMaterial` field of the `Account` record for the account +in Mongodb. + +``` +{ + "AccessKeyId": "...", + "SecretAccessKey": "...", + "Roles": { + "account-short-name": { + "ARN": "arn:aws:iam::...:role/...", + "ExternalId": "..." + } + } +} +``` + +The fields are: + +- `AccessKeyId` which is the access key ID of an IAM user that can assume roles + in the target account. +- `SecretAccessKey` is the secret key for the IAM user identified by the access + key ID. +- `Roles` is a mapping of account `ShortName` to an AWS ARN and ExternalId used + for assuming roles. The `ARN` is the role to be assumed and the `ExternalId` + is the External ID configured in the role trust policy. + +Note that this application does **not** use the AWS Vault backend to assume +roles due to limitations AWS places on roles assumed using assumed role +credentials and not user credentials. + +Finally the broker will need access to a Mongodb credential. Currently this +requires a static credential but in the future a dynamic credential will be +acceptable. Create this credential and pass it in the `--mongodb-vault-path` +argument. It should have the form `database/static-creds/your-cred-name`. + +### Mongodb Setup + +The application needs a Mongodb database with at least a `users` collection +created. The rest are optional. + +To create the `users` collection insert a record to the collection that +contains your GitHub username as the `_id` field and has `IsAdmin` set to true. +For example: + +``` +db.users.insert({ "_id": "mcrute", "IsAdmin": true }) +``` + +This is adequate to allow login via GitHub for the named user. All other fields +will be populated automatically by the app. + +To create allowed AWS accounts for use in the broker add them to the `accounts` +collection. Those records have the following form: + +``` +{ + "_id": "acocunt-short-name", + "AccountNumber": 12345, + "AccountType": "aws", + "ConsoleSessionDuration": 21600000000000, + "DefaultRegion": "us-west-2", + "Name": "Some Description", + "Users": [ + "username1", + "username2" + ], + "VaultMaterial": "some/path/as/above" +} +``` + +- `_id` is the short name of the account. This must also map into a Vault + material per above. +- `AccountNumber` is the AWS account number for the account being brokered. +- `AccountType` is the string `aws` for now +- `ConsoleSessionDuration` is the number of nanoseconds a console session can + exist before timing out. (Nanoseconds are silly, yes; it's a serialized + time.Time) +- `DefaultRegion` is the region considered default, this is just used as an + indicator in the JSON API. +- `Name` a longer form, descriptive name of the account. +- `Users` a case-sensitive list of GitHub user names that have access to this + account +- `VaultMaterial` the path to the Vault material that contains the account + credentials document, as above. + +## To Do + +- Allow GitLab CI jobs to auth using [job tokens](https://docs.gitlab.com/ee/api/jobs.html#get-job-tokens-job) +- Implement an Admin UI, all admin ops are directly on the DB at the moment +- Support dynamic mongodb credentials +- Support for other clouds + - GCP + - OCI + - Azure + +## Contributors + +- [Mike Crute](https://mike.crute.us) diff --git a/README_BROKER.md b/README_BROKER.md new file mode 100644 index 0000000..03db1af --- /dev/null +++ b/README_BROKER.md @@ -0,0 +1,228 @@ +# AWS Identity Broker + +The identity broker is used to obtain short-lived and per-region credentials +for an account. Opt-in regions require the use of a long-lived credential (e.g. +IAM user), enabling global STS tokens, or an STS token sourced in that region. +The identity broker holds long-term credentials and uses them to acquire +short-term credentials in a given region. The broker also provides a list of +opt-in regions and should be used to enumerate regions. + +For human-interactive users the identity broker performs OAUTH against GitHub +to chain the user's GitHub identity to the tokens they are given from the +broker. The broker also provides the user an API key to use when interacting +with the broker programmatically. + +# The API + +The identity broker API is a REST-ful service with an entry-point of +`/api/account`. All further navigation through the API follows links within the +hypertext. + +**Note:** Outside of the account entry-point, URI formats should be considered +opaque implementation details of the broker API and should never be templated. +Beyond the account entry-point, nothing in this specification is normative with +respect to URI paths and locations. + +## Media Formats + +There are two supported media types in the API `application/vnd.broker.v1+json` +and `application/vnd.broker.v2+json`. A client that requests the default media +type or `application/json` will recieve the V1 media type. + +To request a specific media type use the `Accept` header: + +``` +Accept: application/vnd.broker.v2+json +``` + +## Authentication + +All requests to the API must be authenticated with a broker-specific key. That +key is provided to the broker in the `Authorization` header with a sub-type of +`Bearer`. When the broker determines that the key is either expired or invalid +it must redirect the user to `/logout` to indicate that the user is logged out +and must log-in again. + +API keys are bearer tokens and thus must only be exchanged over HTTPS. + +Example of authorization header: + +``` +Authorization: Bearer ...an auth token... +``` + +Legacy versions of the API supported an `X-API-Key` header, which is now +deprecated but remains supported for backwards compatability. + +## Status Codes + +`200 OK`: indicates that the request to the broker was successful. + +`302 Found`: indicates that the broker is providing a redirect. Users should +check the redirect, if it is to the location `/logout` the user should consider +themselves logged out and proceed to login. This condition should not be +followed. Otherwise the user should follow all redirects. + +`400 Bad Request`: indicates that some part of the request is invalid as +submitted. The hypertext MAY provide a description of this error and how to +remedy it. + +`401 Unauthorized`: indicates that the authentication or authorzation for the +user has failed. This is not retryable, a user should obtain a new +authorization token before attempting another request. + +`429 Rate Limit Exceeded`: indicates that the broker has rate-limited the user. +A user should discontinue requests to the broker immediately and wait for at +least 30 seconds before continuing their requests. The rate limit parameters +are specific to the broker and not controlled by this spec. + +`500 Server Error`: indicates a server error condition that is not under the +user's control. + +## Account End-point + +### V1 Media Type + +The account end-point acts as a index of the rest of the API. It presents a +list of accounts to which the user has access as well as links to navigate +further into the API. The format of this document is: + +`short_name` (string): a url-safe name for the account, used as the primary +account identifier within the broker. + +`account_number` (integer): the AWS account number + +`name` (string): a user-friendly name for the account + +`vendor` (string): the name of the cloud vendor (currently only `aws`) + +`console_redirect_url` (uri): a URI that, when followed, leads to a resource +that redirects the user to an authenticated console session. + +`get_console_url` (uri): a URI that, when followed, leads to a console URL +resource. + +`credentials_url` (uri): a URI that, when followed, leads to a region list +resource. + +`global_credential_url` (uri): a URI that, when followed, leads to a credential +resource which provides a credential usable by all non-opt-in regions. The +contents of this resource are a STS global credential which is not usable in +opt-in regions. + +``` +[ + { + "short_name": "primary-account", + "vendor": "aws", + "account_number": 123456789012, + "name": "Primary AWS Account", + "console_redirect_url": "https://broker/api/account/primary-account/console?redirect=1", + "get_console_url": "https://broker/api/account/primary-account/console", + "credentials_url": "https://broker/api/account/primary-account/credentials", + "global_credential_url": "https://broker/api/account/primary-account/credentials/global" + } +] +``` + +### V2 Media Type + +The fiends in the V2 media type have identical meaning to those in the V1 media +type but the top level container format is different. Instead of containing a +list of accounts with a `vendor` field V2 responses are a map of vendors to a +list of accounts. + +``` +{ + "aws": [ + { + "short_name": "primary-account", + "account_number": 123456789012, + "name": "Primary AWS Account", + "console_redirect_url": "https://broker/api/account/primary-account/console?redirect=1", + "get_console_url": "https://broker/api/account/primary-account/console", + "credentials_url": "https://broker/api/account/primary-account/credentials", + "global_credential_url": "https://broker/api/account/primary-account/credentials/global" + } + ] +} +``` + +## Console URL Resource + +**Note:** This resource is not used by the build scripts. + +The console URL resource provides a URL to the AWS console. This resource is +designed for interactive use. + +When provided the query parameter `redirect` with a value of `1` this resource +will not generate a body and will instead redirect to the URL that would have +been returned for `console_url`. + +`console_url` (uri): a link to the AWS console with authentication credentials +embedded. + +``` +{ + "console_url": "https://signin.aws.amazon.com/federation?..." +} +``` + +## Credential Resource + +The credential resource provides a set of credentials that can be used to +configure AWS tools to access an account. + +`access_key` (string): the AWS access key ID + +`secret_key` (string): the AWS secret access key + +`session_token` (string): the AWS session token + +`expiration` (iso-formatted date): the date and time when the credential will +expire + +``` +{ + "access_key": "ASIA123ABC456DEF567G", + "secret_key": "r7KcIuGdPwoUG2YOLISX2XDrVts55IFGTGaY5Tqa", + "session_token": "7C7FyvzyneaS/eRCVDcjHOSTTIHQyvhGqW...", + "expiration": "2020-01-01T00:00:00Z" +} +``` + +The API will set a valid `Expires` header that matches the `expiration` field +of this resource to facilitate use of HTTP caches. + +## Region List Resource + +The region list resource provides a list of regions associated with the account +both opted-in and not. For opted-in regions the resource includes a link to a +credential resource for that region. + +`name` (string): AWS name for the region + +`enabled` (boolean): indicates if the region is enabled and opted-in for this +account. + +`credentials_url` (uri): a URI that, when followed, leads to a credential +resource containing a credential for access to that region. The credential +provided will be usable against the region-local STS endpoint for the specified +region. This also applies for classic regions, which typically use a global +endpoint and credential. The returned credential is scoped to the acquiring +region and may not be usable against the global endpoints or a different +regional endpoint. + +``` +[ + { + "name": "af-south-1", + "enabled": false + }, + { + "name": "us-west-2", + "enabled": true, + "credentials_url": "https://broker/api/account/primary-account/credentials/us-west-2" + } +] +``` -- cgit v1.2.3