Detailed Guide to Service Creation

For quickly getting started with Service Creation in OpsLevel, check out our Getting Started Guide first. This guide provides in-depth information about the feature.

What is Service Creation?

Setting up a new service can be a very daunting and time-consuming task for developers. Developers excel at writing software, but setting up the scaffolding and infrastructure for a brand new service involves operations and decisions that are more in the infrastructure-focused platform and/or SRE teams’ wheelhouse. Depending on how the organization is set up, there are a number of tasks involved with service scaffolding that take developers away from delivering software: long sequence of manual operations to go through, waiting on time/support from the resident platform team, etc.

This is the problem that Service Creation solves. With Service Creation, Platform/SRE teams are able to define reusable templates that contain all the necessary scaffolding to set up a new service. Developer teams can then use these templates by entering in template variables to configure their service. This accelerates the pace at which developer teams can get started, while allowing platform teams to encode best practices and organizational standards into the services right from initial creation.

How does Service Creation in OpsLevel work?

Service Creation: How it works in OpsLevel

The diagram above depicts how service creation works under-the-hood in OpsLevel.

1. Define Templates: Platform/SRE teams can encode service creation boilerplate and best practices in the form of Cookiecutter templates. Templates can be saved in private git repositories, or available from public community repositories in GitHub.
2. Import Templates into OpsLevel: These templates can be imported into OpsLevel by Admin users to provide engineering teams with a standard set of templates to choose from when creating a new service.
3. Trigger service creation from templates: To spin up a new service, a developer will select a template and enter inputs that specifies their service’s required configuration.
4. OpsLevel creates new service and repository: In the backend, OpsLevel:
5. runs the input values through the template and generates the files for the new service
6. initializes a new repository for the service and creates an initial pull/merge request against it adding the template-generated files
7. creates a new OpsLevel Service inside OpsLevel’s Service Catalogue
8. associates the new repository with the new service entry inside OpsLevel

Presently, OpsLevel supports directly creating a new service repository in GitHub or GitLab. For other git forges like Azure DevOps and Bitbucket, we provide the ability to download the generated files which can then be uploaded to the service's repository. We will gradually introduce support for other git forges.

Cookiecutter

There are various tools out there that enable you to create code templates. One of the more popular ones is Cookiecutter.

At a high level, the Cookiecutter workflow looks like this:

1. It allows you to use Jinja templating syntax to compose templated code
2. There is a cookiecutter.json file that lists all the variables used within the template. The JSON files include default values and/or options for each of those variables.
3. It provides a command-line interface that either interactively prompts you for inputs for those variables or you can feed a preset list of variable values, and it uses those with the templated code to generate the specific set of files relevant to your particular project
4. It even allows you to define actions to take before or after the template files are generated in the form of pre- and post-generate hooks, which are essentially Python scripts to run before and after template generation respectively

OpsLevel supports the use of Cookiecutter templates for service creation. It accepts template input values from the developer then generates the new service by writing out the files to a new service repository and adding the entry to service catalogue in OpsLevel.

See the official Cookiecutter documentation to learn how to create your own Cookiecutter templates.

Setting up to use Service Templates and Creation

When using Service Creation, there are two independent interactions with git forges:

1. Importing templates from a Cookiecutter repository hosted in some git forge
2. Writing out the generated files for the new service to a new repository in a git forge

Support for importing templates

For importing templates, we support importing Cookiecutter templates hosted in:

• The user’s private repositories in GitHub or GitLab. To allow this to happen, OpsLevel must be first granted access to the git forge in question by setting up the relevant OpsLevel git integration first.
• Public repositories hosted in GitHub. GitHub is the most popular choice for community-hosted Cookiecutter templates, so OpsLevel allows you to import templates from public repositories there for use to generate your own services.

Support for service creation output

The second interaction with git forges involves actually writing out the files to a new repository in the organization’s git forge. To successfully do this, you will need to have set up a git integration with OpsLevel with your organization’s git forge.

• GitHub, Gitlab, and Azure DevOps - Create and write to new repository: At present, we support creation of a new service repository and writing out the generated files for the service directly to that repository (in a new pull request). To successfully to create the new repository, OpsLevel will require access to your organization’s git forge via the selected OpsLevel Git integration.
• Other git forges - Download service files: We do not yet support creating a new repository with Bitbucket and other git forges. Instead, you can download the generated service files in a compressed tarball (\*.tar.gz). The developer will then have create a new service repository themselves and upload the generated files there. NOTE: We have plans to roll out support of the repository creation flow for these other git forges in the future.

Managing Service Templates (Admins Only)

Only users with an Admin role are allowed to manage service templates (i.e., importing, editing and deleting templates).

Importing Service Templates

New templates can be imported in the main Service Templates page, which can be found in the main menu under “Self-Service > Create New Service”. Once there, you can click the “Import Template” button to start the template import flow.

To import a new template, select the “Import template” from the top-right hand corner. This will bring up a modal requesting the URL to the cookiecutter.json file of the Cookiecutter template you wish to import. Please be sure to enter the URL to the actual cookiecutter.json file and not just the template repo URL.

Once a valid cookiecutter.json filepath has been entered, you are asked to enter metadata about the template such as name, description, language and framework. This information will be shown when listing the imported template in the template gallery.

Send a webhook post-Service Creation

OpsLevel supports the ability to send a webhook after a Service Template is run. A webhook can be used to kick-off a number of workflows that will aid in reducing the amount of time it takes developers to get their services up-and-running such as:

• Paving Infrastructure
• Kicking off a CI/CD Pipeline
• Send Slack Alerts

When creating or editing a Service Template, you can add a webhook by selecting the "Send a webhook post-service creation" checkbox.

Once selected you will be given the ability to configure a webhook that will fire once your Service Template is run.

Configuring the webhook

When creating a webhook via Service Templates, you are able to define the URL where the request will be sent and any information included in the payload or headers:

When you're happy with your webhook you can click Save which will import the template.

Once imported, you and others in the organization will be able to see the template in the Service Templates gallery.

The template is now ready for others to use to spin up a new service.

Editing or deleting service templates

You can edit or delete service templates from the gallery.

Select “Edit” to edit the details of a service template.

Select “Delete” and confirm deletion to remove the service template from your gallery. Note that any services generated from that template will not be deleted.

Creating new services

We presently support two flows for service creation:

1. Directly write new service repository to your git forge (GitHub, GitLab, and Azure DevOps supported, Bitbucket support coming soon)
2. Allow download of service files generated using the template, requiring manual creation of the service repository by the user

Eventually, we will directly write to all git forge integrations, and the download of files will no longer be required. But until then, we will support the first flow and fall back to the second depending on what your organization is set up for.

Choose service template

The first step is to choose the service template from all the templates imported into the account by your account administrators. You can view these by navigating to “Self-Service > Create New Service” from the main menu.

Once there, you will be able to see all the imported templates along with their names, descriptions and any other language/framework details that have been provided. From this page, click the “Create New Service” link on the template you wish to use.

Directly create new service repository (GitHub, GitLab, and Azure DevOps only for now)

This section applies only for accounts that have set up the OpsLevel GitHub, GitLab, or Azure DevOps integration. If your organization uses any other git forge for saving service repositories, please go to the next section to use the manual download flow.

If your organization uses GitHub, GitLab, or Azure DevOps to store service repositories and has the OpsLevel GitHub, GitLab, or Azure DevOps integration set up, you can create the service repository and add the new service files directly from within OpsLevel’s Service Creation feature.

When you have selected “Create New Service” from the Service Templates page (see previous step), you will be presented with a 3-step form. The first step will ask for details about the new service repository:

• Which GitHub, GitLab, or Azure DevOps integration to write to
• What to name the repository
• Whether the repository should be private or public

In the next step, you will be prompted for inputs into the template. These inputs are derived from parsing the cookiecutter.json file itself - as you can see in the image below, we translate the cookiecutter.json file into a form. We show the default values for all the input fields. Any inputs where you are to select one of multiple options, we show a dropdown. And if there are any hidden inputs (i.e., for template-internal use), we do not show those.

Once you have customized the template variables specific to your new service’s needs, you can move onto the third and final step. This is where you define how the service should appear in OpsLevel - the service name, description and owner.

With all the steps completed, the service creation can now begin. You will see a screen showing ongoing progress of the service creation process. As it progresses, you will see the log lines appear from the backend service creation process that is running Cookiecutter.

If there are any issues with generating the service, you will see the screen report a failure to create the new service. The logs should help you troubleshoot the problem. It may be a problem with the template itself, bad input values to the template, or an error in the backend for OpsLevel. If there are any issues with service creation that indicate an issue on our end, please notify your OpsLevel account administrator or contact our support to help you resolve the issue.

If the service creation is successful, you will see the screen reporting success and providing you two buttons.

Click “View Pull Request” to visit the pull or merge request adding service files to the newly created service repository. As the developer, you can review the initial boilerplate being added for your service, and if required, make some adjustments, and then merge it into your new service repository.

Click “View Service Name” to navigate to the service entry within OpsLevel itself. The service will have all the metadata you provided in the last step of the service creation form, and also include common metadata from the service template itself like language and framework. Also, the newly created repository will be linked to it.

You have now successfully generated the initial boilerplate required for a new service in your organization, with all the metadata stored within the service catalogue in OpsLevel!

Templates with a webhook step

If the template you're running has a webhook attached, you will see a second step begin to run once your new Service has been created. Upon successful completion the user will see the Response Message for the webhook that was configured in the template.

Download service files

If your organization uses Bitbucket or another non-supported git provider, you can still create new service files using a template. We will gradually roll out support for all other git forges in the future. If your organization uses GitHub, GitLab, or Azure DevOps as the main git forge, please see section above on how to create a new service repository directly in GitHub or GitLab.

If we don’t yet support directly writing service repositories to your organization’s git forge, the fallback for service creation is to download the service files, then manually create the service repository and upload the files there.

With the download flow, after clicking “Create New Service” from the gallery of Service Templates (see previous step), you will see a two-step form. In the first step, you will be prompted for inputs into the template. These inputs are derived from parsing the cookiecutter.json file itself - as you can see in the image below, we translate the cookiecutter.json file into a form. We show the default values for all the input fields. Any inputs where you are to select one of multiple options, we show a dropdown. And if there are any hidden inputs (i.e., for template-internal use), we do not show those.

Once you have customized the template variables specific to your new service’s needs, you can move onto the second and final step. This is where you define how the service should appear in OpsLevel - the service name, description and owner.

Download Flow Step 2: OpsLevel Service metadata

With all the steps completed, the service creation can now begin. You will see a screen showing ongoing progress of the service creation process. As it progresses, you will see the log lines appear from the backend service creation process that is running Cookiecutter.

If there are any issues with generating the service, you will see the screen report a failure to create the new service. The logs should help you troubleshoot the problem. It may be a problem with the template itself, bad input values to the template, or an error in the backend for OpsLevel. If there are any issues with service creation that indicate an issue on our end, please notify your OpsLevel account administrator or contact our support to help you resolve the issue.

If the service creation is successful, you will see the screen reporting success and providing you two buttons. Click “Download File” to download the service files in a compressed tarball (\*.tar.gz extension). Instructions on how to manually set up the service repository will appear under “Next Steps”.

Click “View Service Name” to navigate to the service entry within OpsLevel itself. The service will have all the metadata you provided in the last step of the service creation form, and also include common metadata from the service template itself like language and framework. Also, the “Next Steps” prompt in the service creation completion screen will show up here. It will have the “Download File” option here as well to allow you to download the generated service files. Once you have set up and linked a service repository with this service, this “Next Steps” prompt will no longer show up.

The next steps to create the initial service repository will have to be manually done by you:

1. You can create the new service repository in the your organization’s git forge, and create an initial pull/merge request against that repository adding the files downloaded from OpsLevel. Once reviewed, the merge request can be merged in.
2. Connect the newly created repository with the new service in OpsLevel

Access Control

There are two primary actions involved with the Service Creation feature:

1. Managing templates: importing, editing and deleting templates
2. Defining and managing the templates for new services would usually be the responsibility of an organization’s Platform/SRE team, who also usually are administrators of the organization’s OpsLevel setup. Thus, we only allow users with an Admin role to manage templates.
3. Using templates: creating new services from already imported templates
4. This action is meant to be available for developer teams in the organization, who would usually be regular (non-admin) users in OpsLevel. They will have view-only access to templates and be able to actually use them to create new services, but not be able to edit the set of available templates.

Troubleshooting

Here are a few tips for common issues you may run into while using the Service Creation feature.

1. Ensure integration is set up for importing templates from private repos: We support importing Cookiecutter templates from your organization’s private repositories, but OpsLevel needs to first be granted access to those repositories. If you encounter errors in importing a private repository, please first check that there is an active integration set up in your OpsLevel account against the git forge hosting the template you are trying to import.
2. Do not use Cookiecutter's Pre/Post-Generate Hooks that try talking to external resources: Cookiecutter supports running pre/post-generate hooks, which are Python scripts that can run before or after the service template files are generated. OpsLevel will attempt to run these scripts if they exist, but if the scripts are trying to connect to some external service, they will fail because we are running the service-creation logic in an isolated environment. OpsLevel allows you to set-up a post-template hook that can be configured within the Service Template. See instructions above.
3. Check the cookiecutter logs: OpsLevel will show you all the logs of what is happening when we run cookiecutter with the provided inputs against the chosen template. You can use these logs to troubleshoot whether there is an issue with the template itself or if an invalid input was entered. If there is an issue with the template, update the template in its repo, then re-import into OpsLevel and try again. If it’s an invalid input, retry service creation with the same template and adjust the faulty values.
4. Contact support: If you encounter failures with service creation that aren’t readily addressed by the suggestions above, please get in touch with our support (through your organization’s OpsLevel account admin) and we will work with you to resolve the issue.