This guide will walk you through how to setup your self hosted OpsLevel application and the prerequisites required to start the application.
- A license file provided by OpsLevel
- A place to host the application (VM or Kubernetes cluster). The OpsLevel Application requires the following minimum specifications:
- 20 CPU
- 40 GB of Memory
- 120 GB of Disk Storage
- Admin access in various third party software to setup integrations
The Self Hosted OpsLevel application is mostly configured using Kubernetes. We recommend ensuring that you have an in-house expert on your internal kubernetes capabilities when setting up your self hosted OpsLevel.
Your OpsLevel account manager will provide your license and instructions for how to perform the installation depending on if you are using a Kubernetes Cluster (preferred) or VM to host the application.
To learn more about the installation methods that Replicated supports, check out their documentation here. During and after installation, you'll have access to the self hosted OpsLevel Admin Console and Configuration page to manage the lifecycle of the self hosted OpsLevel application. This console is powered by replicated and you can see more details by reviewing their documentation here.
Once you have performed the installation and you have logged into the customer admin console, you will be asked to provide your license file before you can begin any of the configuration.
If you have provided a valid license you will then be prompted with a configuration screen to supply any additional configuration for connecting to an external MySQL DB or integrating with third party software like GitHub, Jira, PagerDuty, etc.
Pilot or Production?
In order to make it easy to get started with Pilots and Proof of Concepts, OpsLevel has provided limited default infrastructure for MySQL, Redis and Elasticsearch. For these use cases, you can complete your installation after following the Account Configuration, SMTP Configuration and Runner Configuration sections.
Before your production rollout, we recommend configuring your own infrastructure components as well.
All self hosted OpsLevel applications will follow these steps to get started:
- Create an initial user and configure the domain that users will use to access the OpsLevel Application.
- (Optional, but recommended) Enable email by setting up the SMTP Configuration.
- Deploy your configuration changes and login to the OpsLevel Application.
- (Optional, but recommended) Setup your OpsLevel Runner configuration and redeploy your configuration.
- Setup your initial set of users via SSO, SCIM or manual invitations.
- Start integrating to build out your catalog.
The Account Configuration section allows you to configure an initial Admin user and define how your users will navigate to OpsLevel while on your network.
The initial OpsLevel User should be considered a throwaway user, so feel free to use the default configuration we provide. You may modify the User's name, email and password if you prefer to override these settings.
Once the application is running, you can login with the initial user credentials and proceed with adding additional users to OpsLevel. You can do this by:
Before removing the initial user, ensure that you've promoted at least one of your real users to the Admin role. Once the initial user is removed, it cannot be restored via the KOTS Admin panel.
Before you invite users via the UI or API, you'll need enable your OpsLevel application to send emails by using the SMTP Configuration. Once this is configured, users will be able to receive invites and welcome emails.
See the Configuring Email via SMTP section below for more details.
The Account Configuration section provides Subdomain and Domain fields which are used by the application to generate URLs that enable users to navigate around the application.
By default, the OpsLevel configuration does not include creation of any Kubernetes ingress and the service resource is not exposed. So it is up to you use expose the application in whatever way your company see fit with whatever ingress methods you use.
When you expose the application via ingress we recommend that you make sure you set the proxy body and header size limits to at least 4 MB for payload and large webhook ingestion.
The SMTP Configurationsection allows you to configure SMTP settings so that the application may send emails. It is recommended that you give the application access to send email so you may receive the following types of emails:
- User Invite
- Password Reset
- Weekly Report Emails for Service Owners
- Campaign Notifications
After configuring the basics, you'll want to configure an OpsLevel Runner. OpsLevel Runners are used for a variety of asynchronous jobs in OpsLevel including running Repo Grep Checks and Service Detection from Git Integrations.
To configure OpsLevel Runners with self hosted OpsLevel, first you'll need to get your OpsLevel application running,
Ensure you are logged in as an Admin user to create a token with Read and Write permissions.
- Navigate to Integrations > API Tokens
- Click Create API Token
- In the dialog, add a Token Description (For example, "OpsLevel Runner Token") and ensure that Read-only is unchecked. Note: if you cannot uncheck the Read-only toggle, you may not be an Admin user.
- Copy the Token value and store the value somewhere safe for use later. Note: the token will only be visible at this time. If you lose the value, you will need to create a new API token.
- Return to the KOTS Admin site and navigate to the OpsLevel Application's Config page.
- Navigate to the Runner Confguration section.
- Toggle on the Enable OpsLevel Runner? option.
- Enter the Token value from step 4 above into the OpsLevel Runner API Token field.
- (Optional) Override the OpsLevel Runner Max Replicas setting to handle more or less OpsLevel Runner Jobs. Note: You may want to increase the number of replicas if you have many repositories, large monorepos attached to many services or many Repo Grep Checks that you want to run at once.
- Once you are finished with the settings, click Save Config
- Deploy the OpsLevel Application in order to have your settings take effect. During your next deployment, additional pods will be deployed with your OpsLevel Runner configuration.
Note on MySQL included in installation
By default, the installation creates a MySQL container alongside the application for getting started quickly.
We do not recommend using the included MySQL container in production deployments of OpsLevel! This section provides you with the detailed configuration needed to allow the application to connect to an “external” MySQL database that has been hardened for production and is monitored by you.
The MySQL database is where all of your application data is stored. If this database is destroyed, all of the data within OpsLevel is also destroyed. To prevent data loss, we recommend you setup regular backups.
There are a handful of MySQL parameters that need to be set that we have enumerated here, but do also note that we have configured preflight checks to ensure the database OpsLevel connects to will work properly.
[mysqld] init_connect='SET collation_connection = utf8mb4_unicode_ci,NAMES utf8mb4' character_set_server = 'utf8mb4' collation_server = 'utf8mb4_unicode_ci' [client] default_character_set = 'utf8mb4'
Our application is configured to perform any pending migrations on this database at startup. Please be aware that this is not an “online” process, and as such may lock the database for a period of time while the migrations complete, thereby preventing use of the application. Note: this is something we are looking into changing in the future.
Note on Redis included in installation
By default, the installation creates a Redis container alongside the application for the purpose of getting started quickly.
We do not recommend using the included Redis container in production deployments of OpsLevel!
This section provides you with the configuration needed to allow the application to connect to an “external” Redis instance that has been hardened for production and is monitored by you.
The Redis database is used to store pending asynchronous jobs waiting to be processed. If this database is destroyed all that is lost is pending jobs. For the most part many asynchronous tasks will be re-enqueued at a later time, but there are a few cases where losing the job means that specific data that was ingested will never be processed.
Note on Elasticsearch included in installation
By default, the installation creates an Elasticsearch container alongside the application for the purpose of getting started quickly.
We do not recommend using the included Elasticsearch container in production deployments of OpsLevel!
This section provides you with the configuration needed to allow the application to connect to an “external” Elasticsearch instance that has been hardened for production and is monitored by you.
This section allows you to connect an Elasticsearch cluster to OpsLevel to power features related to improved Search and Tech Docs.
OpsLevel expects that your Elasticsearch cluster version is
8.4.2 or higher. If you don't have an Elasticsearch cluster available, we recommend using a deployment provided by Elastic.co directly.
- Protocol - WARNING: always select
- Username - usually
- Endpoint Hostname - something like
- Port Number - usually
OpsLevel bundles a single-node Elasticsearch cluster on version
8.8.2. As mentioned above, this is only intended to be used for testing and is not suitable for production!
- Protocol - select
elasticsearch-0Pod lives on the same network as the OpsLevel
- Username -
- Endpoint Hostname -
- Port Number -
- Kubernetes Storage Class -
- Elasticsearch Storage Size (Gi) -
You'll notice that the OpsLevel configuration has many more sections than those that we've walked through above. Most of the additional settings are related to specific integrations. Many of the integrations can be configured from within the OpsLevel application using Personal Access Tokens or other authentication tokens, however some integrations require that additional applications be configured within the 3rd Party Service.
Integrations that require 3rd party applications and updates to your OpsLevel Application configuration include:
For more information about how to perform the advanced integration configurations, see our list of guides here.
Once you have completed all the needed configurations you can click Next and the preflight checks will begin to run. The purpose of these checks is to ensure the provided configuration and application hosting location have the best chance of starting up correctly. If any of the preflight checks fail you will need to fix them before completing the installation.
Updated about 2 months ago