Install RH-SSO on Openshift
#google , #rh-sso , #keycloak , #ansible , #authentication , #authorization
Install RH-SSO on Openshift
First create a new project to keep RH-SSO work clean and easily delete it if necessary.
oc new-project ssoI started by exporting the existing RH-SSO persistent depoyment template. This builds up one RH-SSO app with one postgres persistent database by default. You’ve got lots of other choices.
$ oc get template -n openshift | grep sso
eap64-sso-s2i An example EAP 6 Single Sign-On application. For more information about using... 44 (19 blank) 8
eap70-sso-s2i An example EAP 7 Single Sign-On application. For more information about using... 44 (19 blank) 8
eap71-sso-s2i An example EAP 7 Single Sign-On application. For more information about using... 44 (19 blank) 8
sso71-https An example SSO 7 application. For more information about using this template,... 26 (15 blank) 6
sso71-mysql An example SSO 7 application with a MySQL database. For more information abou... 36 (20 blank) 8
sso71-mysql-persistent An example SSO 7 application with a MySQL database. For more information abou... 37 (20 blank) 9
sso71-postgresql An example SSO 7 application with a PostgreSQL database. For more information... 33 (17 blank) 8
sso71-postgresql-persistent An example SSO 7 application with a PostgreSQL database. For more information... 34 (17 blank) 9
sso72-https An example SSO 7 application. For more information about using this template,... 26 (15 blank) 6
sso72-mysql An example SSO 7 application with a MySQL database. For more information abou... 36 (20 blank) 8
sso72-mysql-persistent An example SSO 7 application with a MySQL database. For more information abou... 37 (20 blank) 9
sso72-postgresql An example SSO 7 application with a PostgreSQL database. For more information... 33 (17 blank) 8
sso72-postgresql-persistent An example SSO 7 application with a PostgreSQL database. For more information... 34 (17 blank) 9These templates are all ready to use by just passing in variables, but I like to customize things as I work with them. As of this writing, you can see the template as below, with variables at the bottom:
$ oc export template sso72-postgresql-persistent -n openshift > base-template.yml
$ cat base-template.yml
apiVersion: v1
kind: Template
labels:
template: sso72-postgresql-persistent
xpaas: 1.4.9
message: 'A new persistent SSO service (using PostgreSQL) has been created in your
project. The admin username/password for accessing the master realm via the SSO
console is ${SSO_ADMIN_USERNAME}/${SSO_ADMIN_PASSWORD}. The username/password for
accessing the PostgreSQL database "${DB_DATABASE}" is ${DB_USERNAME}/${DB_PASSWORD}.
Please be sure to create the following secrets: "${HTTPS_SECRET}" containing the
${HTTPS_KEYSTORE} file used for serving secure content; "${JGROUPS_ENCRYPT_SECRET}"
containing the ${JGROUPS_ENCRYPT_KEYSTORE} file used for securing JGroups communications;
"${SSO_TRUSTSTORE_SECRET}" containing the ${SSO_TRUSTSTORE} file used for securing
SSO requests.'
metadata:
annotations:
description: An example SSO 7 application with a PostgreSQL database. For more
information about using this template, see https://github.com/jboss-openshift/application-templates.
iconClass: icon-sso
openshift.io/display-name: Single Sign-On 7.2 + PostgreSQL
openshift.io/provider-display-name: Red Hat, Inc.
tags: sso,keycloak,jboss
template.openshift.io/documentation-url: https://access.redhat.com/documentation/en/red-hat-single-sign-on/
template.openshift.io/long-description: This template defines resources needed
to develop Red Hat Single Sign-On 7.2 server based deployment and deployment
configuration for PostgreSQL using persistence.
template.openshift.io/support-url: https://access.redhat.com
version: 1.4.9
creationTimestamp: null
name: sso72-postgresql-persistent
objects:
- apiVersion: v1
kind: Service
metadata:
annotations:
description: The web server's http port.
service.alpha.openshift.io/dependencies: '[{"name": "${APPLICATION_NAME}-postgresql",
"kind": "Service"}]'
labels:
application: ${APPLICATION_NAME}
name: ${APPLICATION_NAME}
spec:
ports:
- port: 8080
targetPort: 8080
selector:
deploymentConfig: ${APPLICATION_NAME}
- apiVersion: v1
kind: Service
metadata:
annotations:
description: The web server's https port.
service.alpha.openshift.io/dependencies: '[{"name": "${APPLICATION_NAME}-postgresql",
"kind": "Service"}]'
labels:
application: ${APPLICATION_NAME}
name: secure-${APPLICATION_NAME}
spec:
ports:
- port: 8443
targetPort: 8443
selector:
deploymentConfig: ${APPLICATION_NAME}
- apiVersion: v1
kind: Service
metadata:
annotations:
description: The database server's port.
labels:
application: ${APPLICATION_NAME}
name: ${APPLICATION_NAME}-postgresql
spec:
ports:
- port: 5432
targetPort: 5432
selector:
deploymentConfig: ${APPLICATION_NAME}-postgresql
- apiVersion: v1
kind: Service
metadata:
annotations:
description: The JGroups ping port for clustering.
service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
labels:
application: ${APPLICATION_NAME}
name: ${APPLICATION_NAME}-ping
spec:
clusterIP: None
ports:
- name: ping
port: 8888
selector:
deploymentConfig: ${APPLICATION_NAME}
- apiVersion: v1
id: ${APPLICATION_NAME}-http
kind: Route
metadata:
annotations:
description: Route for application's http service.
labels:
application: ${APPLICATION_NAME}
name: ${APPLICATION_NAME}
spec:
host: ${HOSTNAME_HTTP}
to:
name: ${APPLICATION_NAME}
- apiVersion: v1
id: ${APPLICATION_NAME}-https
kind: Route
metadata:
annotations:
description: Route for application's https service.
labels:
application: ${APPLICATION_NAME}
name: secure-${APPLICATION_NAME}
spec:
host: ${HOSTNAME_HTTPS}
tls:
termination: passthrough
to:
name: secure-${APPLICATION_NAME}
- apiVersion: v1
kind: DeploymentConfig
metadata:
labels:
application: ${APPLICATION_NAME}
name: ${APPLICATION_NAME}
spec:
replicas: 1
selector:
deploymentConfig: ${APPLICATION_NAME}
strategy:
type: Recreate
template:
metadata:
labels:
application: ${APPLICATION_NAME}
deploymentConfig: ${APPLICATION_NAME}
name: ${APPLICATION_NAME}
spec:
containers:
- env:
- name: DB_SERVICE_PREFIX_MAPPING
value: ${APPLICATION_NAME}-postgresql=DB
- name: DB_JNDI
value: ${DB_JNDI}
- name: DB_USERNAME
value: ${DB_USERNAME}
- name: DB_PASSWORD
value: ${DB_PASSWORD}
- name: DB_DATABASE
value: ${DB_DATABASE}
- name: TX_DATABASE_PREFIX_MAPPING
value: ${APPLICATION_NAME}-postgresql=DB
- name: DB_MIN_POOL_SIZE
value: ${DB_MIN_POOL_SIZE}
- name: DB_MAX_POOL_SIZE
value: ${DB_MAX_POOL_SIZE}
- name: DB_TX_ISOLATION
value: ${DB_TX_ISOLATION}
- name: JGROUPS_PING_PROTOCOL
value: openshift.DNS_PING
- name: OPENSHIFT_DNS_PING_SERVICE_NAME
value: ${APPLICATION_NAME}-ping
- name: OPENSHIFT_DNS_PING_SERVICE_PORT
value: "8888"
- name: HTTPS_KEYSTORE_DIR
value: /etc/eap-secret-volume
- name: HTTPS_KEYSTORE
value: ${HTTPS_KEYSTORE}
- name: HTTPS_KEYSTORE_TYPE
value: ${HTTPS_KEYSTORE_TYPE}
- name: HTTPS_NAME
value: ${HTTPS_NAME}
- name: HTTPS_PASSWORD
value: ${HTTPS_PASSWORD}
- name: JGROUPS_ENCRYPT_SECRET
value: ${JGROUPS_ENCRYPT_SECRET}
- name: JGROUPS_ENCRYPT_KEYSTORE_DIR
value: /etc/jgroups-encrypt-secret-volume
- name: JGROUPS_ENCRYPT_KEYSTORE
value: ${JGROUPS_ENCRYPT_KEYSTORE}
- name: JGROUPS_ENCRYPT_NAME
value: ${JGROUPS_ENCRYPT_NAME}
- name: JGROUPS_ENCRYPT_PASSWORD
value: ${JGROUPS_ENCRYPT_PASSWORD}
- name: JGROUPS_CLUSTER_PASSWORD
value: ${JGROUPS_CLUSTER_PASSWORD}
- name: SSO_ADMIN_USERNAME
value: ${SSO_ADMIN_USERNAME}
- name: SSO_ADMIN_PASSWORD
value: ${SSO_ADMIN_PASSWORD}
- name: SSO_REALM
value: ${SSO_REALM}
- name: SSO_SERVICE_USERNAME
value: ${SSO_SERVICE_USERNAME}
- name: SSO_SERVICE_PASSWORD
value: ${SSO_SERVICE_PASSWORD}
- name: SSO_TRUSTSTORE
value: ${SSO_TRUSTSTORE}
- name: SSO_TRUSTSTORE_DIR
value: /etc/sso-secret-volume
- name: SSO_TRUSTSTORE_PASSWORD
value: ${SSO_TRUSTSTORE_PASSWORD}
image: ${APPLICATION_NAME}
imagePullPolicy: Always
livenessProbe:
exec:
command:
- /bin/bash
- -c
- /opt/eap/bin/livenessProbe.sh
initialDelaySeconds: 60
name: ${APPLICATION_NAME}
ports:
- containerPort: 8778
name: jolokia
protocol: TCP
- containerPort: 8080
name: http
protocol: TCP
- containerPort: 8443
name: https
protocol: TCP
- containerPort: 8888
name: ping
protocol: TCP
readinessProbe:
exec:
command:
- /bin/bash
- -c
- /opt/eap/bin/readinessProbe.sh
resources:
limits:
memory: ${MEMORY_LIMIT}
volumeMounts:
- mountPath: /etc/eap-secret-volume
name: eap-keystore-volume
readOnly: true
- mountPath: /etc/jgroups-encrypt-secret-volume
name: eap-jgroups-keystore-volume
readOnly: true
- mountPath: /etc/sso-secret-volume
name: sso-truststore-volume
readOnly: true
terminationGracePeriodSeconds: 75
volumes:
- name: eap-keystore-volume
secret:
secretName: ${HTTPS_SECRET}
- name: eap-jgroups-keystore-volume
secret:
secretName: ${JGROUPS_ENCRYPT_SECRET}
- name: sso-truststore-volume
secret:
secretName: ${SSO_TRUSTSTORE_SECRET}
triggers:
- imageChangeParams:
automatic: true
containerNames:
- ${APPLICATION_NAME}
from:
kind: ImageStreamTag
name: redhat-sso72-openshift:1.0
namespace: ${IMAGE_STREAM_NAMESPACE}
type: ImageChange
- type: ConfigChange
- apiVersion: v1
kind: DeploymentConfig
metadata:
labels:
application: ${APPLICATION_NAME}
name: ${APPLICATION_NAME}-postgresql
spec:
replicas: 1
selector:
deploymentConfig: ${APPLICATION_NAME}-postgresql
strategy:
type: Recreate
template:
metadata:
labels:
application: ${APPLICATION_NAME}
deploymentConfig: ${APPLICATION_NAME}-postgresql
name: ${APPLICATION_NAME}-postgresql
spec:
containers:
- env:
- name: POSTGRESQL_USER
value: ${DB_USERNAME}
- name: POSTGRESQL_PASSWORD
value: ${DB_PASSWORD}
- name: POSTGRESQL_DATABASE
value: ${DB_DATABASE}
- name: POSTGRESQL_MAX_CONNECTIONS
value: ${POSTGRESQL_MAX_CONNECTIONS}
- name: POSTGRESQL_MAX_PREPARED_TRANSACTIONS
value: ${POSTGRESQL_MAX_CONNECTIONS}
- name: POSTGRESQL_SHARED_BUFFERS
value: ${POSTGRESQL_SHARED_BUFFERS}
image: postgresql
imagePullPolicy: Always
livenessProbe:
initialDelaySeconds: 30
tcpSocket:
port: 5432
timeoutSeconds: 1
name: ${APPLICATION_NAME}-postgresql
ports:
- containerPort: 5432
protocol: TCP
readinessProbe:
exec:
command:
- /bin/sh
- -i
- -c
- psql -h 127.0.0.1 -U $POSTGRESQL_USER -q -d $POSTGRESQL_DATABASE -c
'SELECT 1'
initialDelaySeconds: 5
timeoutSeconds: 1
volumeMounts:
- mountPath: /var/lib/pgsql/data
name: ${APPLICATION_NAME}-postgresql-pvol
terminationGracePeriodSeconds: 60
volumes:
- name: ${APPLICATION_NAME}-postgresql-pvol
persistentVolumeClaim:
claimName: ${APPLICATION_NAME}-postgresql-claim
triggers:
- imageChangeParams:
automatic: true
containerNames:
- ${APPLICATION_NAME}-postgresql
from:
kind: ImageStreamTag
name: postgresql:${POSTGRESQL_IMAGE_STREAM_TAG}
namespace: ${IMAGE_STREAM_NAMESPACE}
type: ImageChange
- type: ConfigChange
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
application: ${APPLICATION_NAME}
name: ${APPLICATION_NAME}-postgresql-claim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: ${VOLUME_CAPACITY}
parameters:
- description: The name for the application.
displayName: Application Name
name: APPLICATION_NAME
required: true
value: sso
- description: 'Custom hostname for http service route. Leave blank for default hostname,
e.g.: <application-name>.<project>.<default-domain-suffix>'
displayName: Custom http Route Hostname
name: HOSTNAME_HTTP
- description: 'Custom hostname for https service route. Leave blank for default
hostname, e.g.: <application-name>.<project>.<default-domain-suffix>'
displayName: Custom https Route Hostname
name: HOSTNAME_HTTPS
- description: Database JNDI name used by application to resolve the datasource, e.g.
java:/jboss/datasources/postgresql
displayName: Database JNDI Name
name: DB_JNDI
value: java:jboss/datasources/KeycloakDS
- description: Database name
displayName: Database Name
name: DB_DATABASE
required: true
value: root
- description: The name of the secret containing the keystore file
displayName: Server Keystore Secret Name
name: HTTPS_SECRET
value: sso-app-secret
- description: The name of the keystore file within the secret
displayName: Server Keystore Filename
name: HTTPS_KEYSTORE
value: keystore.jks
- description: The type of the keystore file (JKS or JCEKS)
displayName: Server Keystore Type
name: HTTPS_KEYSTORE_TYPE
- description: The name associated with the server certificate (e.g. jboss)
displayName: Server Certificate Name
name: HTTPS_NAME
- description: The password for the keystore and certificate (e.g. mykeystorepass)
displayName: Server Keystore Password
name: HTTPS_PASSWORD
- description: Sets xa-pool/min-pool-size for the configured datasource.
displayName: Datasource Minimum Pool Size
name: DB_MIN_POOL_SIZE
- description: Sets xa-pool/max-pool-size for the configured datasource.
displayName: Datasource Maximum Pool Size
name: DB_MAX_POOL_SIZE
- description: Sets transaction-isolation for the configured datasource.
displayName: Datasource Transaction Isolation
name: DB_TX_ISOLATION
- description: The maximum number of client connections allowed. This also sets the
maximum number of prepared transactions.
displayName: PostgreSQL Maximum number of connections
name: POSTGRESQL_MAX_CONNECTIONS
- description: Configures how much memory is dedicated to PostgreSQL for caching data.
displayName: PostgreSQL Shared Buffers
name: POSTGRESQL_SHARED_BUFFERS
- description: Database user name
displayName: Database Username
from: user[a-zA-Z0-9]{3}
generate: expression
name: DB_USERNAME
required: true
- description: Database user password
displayName: Database Password
from: '[a-zA-Z0-9]{8}'
generate: expression
name: DB_PASSWORD
required: true
- description: Size of persistent storage for database volume.
displayName: Database Volume Capacity
name: VOLUME_CAPACITY
required: true
value: 1Gi
- description: The name of the secret containing the keystore file
displayName: JGroups Secret Name
name: JGROUPS_ENCRYPT_SECRET
value: sso-app-secret
- description: The name of the keystore file within the secret
displayName: JGroups Keystore Filename
name: JGROUPS_ENCRYPT_KEYSTORE
value: jgroups.jceks
- description: The name associated with the server certificate (e.g. secret-key)
displayName: JGroups Certificate Name
name: JGROUPS_ENCRYPT_NAME
- description: The password for the keystore and certificate (e.g. password)
displayName: JGroups Keystore Password
name: JGROUPS_ENCRYPT_PASSWORD
- description: JGroups cluster password
displayName: JGroups Cluster Password
from: '[a-zA-Z0-9]{8}'
generate: expression
name: JGROUPS_CLUSTER_PASSWORD
required: true
- description: Namespace in which the ImageStreams for Red Hat Middleware images are
installed. These ImageStreams are normally installed in the openshift namespace.
You should only need to modify this if you've installed the ImageStreams in a
different namespace/project.
displayName: ImageStream Namespace
name: IMAGE_STREAM_NAMESPACE
required: true
value: openshift
- description: SSO Server admin username
displayName: SSO Admin Username
from: '[a-zA-Z0-9]{8}'
generate: expression
name: SSO_ADMIN_USERNAME
required: true
- description: SSO Server admin password
displayName: SSO Admin Password
from: '[a-zA-Z0-9]{8}'
generate: expression
name: SSO_ADMIN_PASSWORD
required: true
- description: Realm to be created in the SSO server (e.g. demo).
displayName: SSO Realm
name: SSO_REALM
- description: The username used to access the SSO service. This is used by clients
to create the appliction client(s) within the specified SSO realm.
displayName: SSO Service Username
name: SSO_SERVICE_USERNAME
- description: The password for the SSO service user.
displayName: SSO Service Password
name: SSO_SERVICE_PASSWORD
- description: The name of the truststore file within the secret (e.g. truststore.jks)
displayName: SSO Trust Store
name: SSO_TRUSTSTORE
- description: The password for the truststore and certificate (e.g. mykeystorepass)
displayName: SSO Trust Store Password
name: SSO_TRUSTSTORE_PASSWORD
- description: The name of the secret containing the truststore file (e.g. truststore-secret).
Used for volume secretName
displayName: SSO Trust Store Secret
name: SSO_TRUSTSTORE_SECRET
value: sso-app-secret
- description: The tag to use for the "postgresql" image stream. Typically, this
aligns with the major.minor version of PostgreSQL.
displayName: PostgreSQL Image Stream Tag
name: POSTGRESQL_IMAGE_STREAM_TAG
required: true
value: "9.5"
- description: Container memory limit
name: MEMORY_LIMIT
value: 1GiMy final template is named sso-template.yml. Take a look at the differences. I ran into a problem having some of
these postgres values blank. I plan to check on that with the development team. Postgres sets the defaults just fine,
but if the variables are present with blank values, it has a problem. One further change I made was to remove the default non-SSL deployment options.
I want SSL only and I want to enforce that by default. I tried leaving the basic HTTP variables blank, but that ended up causing failures.
$ diff base-template.yml sso-template.yml
36,51d35
< description: The web server's http port.
< service.alpha.openshift.io/dependencies: '[{"name": "${APPLICATION_NAME}-postgresql",
< "kind": "Service"}]'
< labels:
< application: ${APPLICATION_NAME}
< name: ${APPLICATION_NAME}
< spec:
< ports:
< - port: 8080
< targetPort: 8080
< selector:
< deploymentConfig: ${APPLICATION_NAME}
< - apiVersion: v1
< kind: Service
< metadata:
< annotations:
95,107d78
< id: ${APPLICATION_NAME}-http
< kind: Route
< metadata:
< annotations:
< description: Route for application's http service.
< labels:
< application: ${APPLICATION_NAME}
< name: ${APPLICATION_NAME}
< spec:
< host: ${HOSTNAME_HTTP}
< to:
< name: ${APPLICATION_NAME}
< - apiVersion: v1
219,221d189
< - containerPort: 8080
< name: http
< protocol: TCP
296,301d263
< - name: POSTGRESQL_MAX_CONNECTIONS
< value: ${POSTGRESQL_MAX_CONNECTIONS}
< - name: POSTGRESQL_MAX_PREPARED_TRANSACTIONS
< value: ${POSTGRESQL_MAX_CONNECTIONS}
< - name: POSTGRESQL_SHARED_BUFFERS
< value: ${POSTGRESQL_SHARED_BUFFERS}
360,363d321
< - description: 'Custom hostname for http service route. Leave blank for default hostname,
< e.g.: <application-name>.<project>.<default-domain-suffix>'
< displayName: Custom http Route Hostname
< name: HOSTNAME_HTTPThe magic now is in the parameter file. You could put all that junk on the command line, but why? This solution is so much more elegant:
$ cat sso-params
HOSTNAME_HTTPS=sso.apps.example.com
HTTPS_NAME=sso
HTTPS_PASSWORD=redacted
DB_USERNAME=ssoapp <!--(1)-->
DB_PASSWORD=redacted
DB_DATABASE=keycloak
JGROUPS_ENCRYPT_NAME=jgroups
JGROUPS_ENCRYPT_PASSWORD=redacted
JGROUPS_CLUSTER_PASSWORD=redacted
SSO_ADMIN_USERNAME=admin
SSO_ADMIN_PASSWORD=redacted
SSO_SERVICE_USERNAME=eap-mgmt-user
SSO_SERVICE_PASSWORD=redacted
SSO_REALM=ocp-
Note that I used a hyphen in the username originally as
sso-app. Perhaps some of you may be wiser to postgres and already know that it does not like hyphens in usernames. Alas I wasted some time figuring this out.
Before creating everything in OpenShift with the template and parameters, you need to create the secrets it will use first. RH-SSO requires a keystore for the SSL cert and for the Jgroups cluster communication.
$ keytool -genkeypair -alias sso -keyalg RSA -keystore keystore.jks -storepass redacted -keypass redacted --dname "CN=sso.apps.example.com,OU=openshift,O=example.com,L=City,S=ST,C=US"
Submit your certificate to a Certificate Authority (CA) if you’d like to make it valid to the outside world (i.e. not self-signed). I skipped this step for the moment just to get things working. Import the signed certificate, along with any intermediate certificates.
$ keytool -certreq -keyalg RSA -alias sso -keystore keystore.jks -file sso.csr $ keytool -import -keystore server.keystore -alias intermediateCA -file intermediate.ca $ keytool -import -alias jboss -keystore server.keystore -file server.crt
Export a self-signed certificate from the keystore. You may need this later.
$ keytool -export -alias sso -keystore keystore.jks -file keystore.crt -storepass redacted
Convert the DER encoded certificate to PEM:
$ openssl x509 -in keystore.crt -inform der -outform pem -out keystore.pem
Convert that to a oneliner because it’s easier to add it to your OpenShift config that way:
$ awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' keystore.pem > keystore-one.pem
Create a keystore for your jgroups cluster communication. This is for protecting the communication between nodes when scaled beyond one. Note that this keystore format is JCEKS, a requirement of jgroups.
$ keytool -genseckey -alias jgroups -storetype JCEKS -keystore jgroups.jceks -storepass redacted -keypass redacted
Create an OpenShift secret from both your java keystores.
$ oc secret new sso-app-secret keystore.jks jgroups.jceks secret/sso-app-secret
Now finally create everything else in OpenShift:
$ oc process -f sso-template.yml --param-file sso-params | oc apply -f-
service "secure-sso" created
service "sso-postgresql" created
service "sso-ping" created
route "secure-sso" created
deploymentconfig "sso" created
deploymentconfig "sso-postgresql" created
persistentvolumeclaim "sso-postgresql-claim" createdBe patient as the postgres database must come up before SSO initializes correctly. At this time, the template expects that the SSO deployment may fail and restart until the postgres database service is available. You may see the below error on the SSO node:
Caused by: java.net.NoRouteToHostException: No route to host (Host unreachable)Postgres had me concerned at first too, with regards to persistent volume claims. But after a few more minutes, it got things right.
$ oc get events | grep -i warn
1m 2m 9 sso-1-4xd92 Pod spec.containers{sso} Warning Unhealthy kubelet, infra1.example.internal Readiness probe failed: {
1m 1m 3 sso-1-4xd92 Pod spec.containers{sso} Warning Unhealthy kubelet, infra1.example.internal Liveness probe failed: {
35s 35s 1 sso-postgresql-1-b84rl Pod Warning FailedMount kubelet, infra1.example.internal Unable to mount volumes for pod "sso-postgresql-1-b84rl_sso-test(c4fb9a8c-5382-11e8-b8b5-001a4a16015f)": timeout expired waiting for volumes to attach/mount for pod "sso-test"/"sso-postgresql-1-b84rl". list of unattached/unmounted volumes=[sso-postgresql-pvol]
8s 8s 1 sso-postgresql-1-b84rl Pod spec.containers{sso-postgresql} Warning Unhealthy kubelet, infra1.example.internal Readiness probe failed: sh: cannot set terminal process group (-1): Inappropriate ioctl for device
[bward@gauss sso]$ oc get pvc
NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE
sso-postgresql-claim Bound pvc-c260f6eb-5382-11e8-9603-001a4a160161 1Gi RWO standard 3m
[bward@gauss sso]$ oc get pv
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE
...
pvc-c260f6eb-5382-11e8-9603-001a4a160161 1Gi RWO Delete Bound sso-test/sso-postgresql-claim standard 3m
...When everything is fine and dandy, you will see this on the SSO host:
18:09:03,656 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Red Hat Single Sign-On 7.2.1.GA (WildFly Core 3.0.12.Final-redhat-1) started in 81679ms - Started 661 of 1061 services (761 services are lazy, passive or on-demand)
You should also see similar output here:
$ oc get all
NAME REVISION DESIRED CURRENT TRIGGERED BY
dc/sso 1 1 1 config,image(redhat-sso72-openshift:1.0)
dc/sso-postgresql 1 1 1 config,image(postgresql:9.5)
NAME DESIRED CURRENT READY AGE
rc/sso-1 1 1 1 7m
rc/sso-postgresql-1 1 1 1 7m
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
routes/secure-sso sso.apps.example.com secure-sso <all> passthrough None
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc/glusterfs-dynamic-sso-postgresql-claim 172.30.232.47 <none> 1/TCP 7m
svc/secure-sso 172.30.172.60 <none> 8443/TCP 7m
svc/sso-ping None <none> 8888/TCP 7m
svc/sso-postgresql 172.30.16.62 <none> 5432/TCP 7m
NAME READY STATUS RESTARTS AGE
po/sso-1-wm2wx 1/1 Running 3 7m
po/sso-postgresql-1-c26cm 1/1 Running 1 7mNotice above that it took 3 restarts of SSO before the postgres pod came up correctly.
TODO: add more notes here
Debugging:
[standalone@localhost:9990 /] /subsystem=logging/logger=org.keycloak:add
{"outcome" => "success"}
[standalone@localhost:9990 /] /subsystem=logging/logger=org.keycloak:write-attribute(name=level,value=DEBUG)
{"outcome" => "success"}
[standalone@localhost:9990 /] /subsystem=logging/logger=org.keycloak:read-resource(recursive=true)
{
"outcome" => "success",
"result" => {
"category" => "org.keycloak",
"filter" => undefined,
"filter-spec" => undefined,
"handlers" => undefined,
"level" => "DEBUG",
"use-parent-handlers" => true
}
}