Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for pipelines controller statefulset ordinals #2361

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add support for pipelines controller statefulset ordinals #2361

wants to merge 1 commit into from
+242 −4

Conversation

jkhelil
Copy link
Member

@jkhelil jkhelil commented Sep 30, 2024 •
edited
Loading

Changes

To handle HA on pipelines controller, there are two mechanisms available for scaling controllers horizontally via knative/pkg:

  • Using leader election, which allows for failover, but can result in hot-spotting.(this is the default pipelines controller mode)
  • Using StatefulSet ordinals, which doesn't allow for failover, but guarantees keys are evenly spread across replicas.
    This PR adds support for pipelines controller statefulset ordinals,
  • it exposes statefulset-ordinals: false by default on tektonconfig
  • When statefulset-ordinals set to true, it creates a new installerset pipeline-main-statefulset-xxx
  • Ensures pipelines controller is deployed as statefulsetSet and has the configuration to handle loadbalancing

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

See the contribution guide for more details.

Release Notes

@tekton-robot tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Sep 30, 2024
@tekton-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign concaf after the PR has been reviewed.
You can assign the PR to them by writing /assign @concaf in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 30, 2024
@jkhelil jkhelil force-pushed the statefulset_ordinals branch 2 times, most recently from 7e1fc2c to 817fd30 Compare September 30, 2024 10:42
@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 93.5% 0.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.0% 1.8
pkg/reconciler/common/transformers.go 74.4% 70.3% -4.1
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 93.5% 0.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.0% 1.8
pkg/reconciler/common/transformers.go 74.4% 70.2% -4.2
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 93.5% 0.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.0% 1.8
pkg/reconciler/common/transformers.go 74.4% 70.4% -4.0
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 93.5% 0.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.0% 1.8
pkg/reconciler/common/transformers.go 74.4% 70.4% -4.0
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 93.5% 0.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.0% 1.8
pkg/reconciler/common/transformers.go 74.4% 70.4% -4.0
pkg/reconciler/kubernetes/tektoninstallerset/client/cleanup.go 61.4% 46.6% -14.8
pkg/reconciler/kubernetes/tektoninstallerset/client/create.go 77.6% 76.2% -1.4
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 93.5% 0.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.0% 1.8
pkg/reconciler/common/transformers.go 74.4% 70.4% -4.0
pkg/reconciler/kubernetes/tektoninstallerset/client/cleanup.go 61.4% 46.6% -14.8
pkg/reconciler/kubernetes/tektoninstallerset/client/create.go 77.6% 76.2% -1.4
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@jkhelil jkhelil changed the title WIP add support for pipelines controller statefulset ordinals Add support for pipelines controller statefulset ordinals Oct 1, 2024
@tekton-robot tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 1, 2024
@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 90.1% -3.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.0% 1.8
pkg/reconciler/common/transformers.go 74.4% 70.4% -4.0
pkg/reconciler/kubernetes/tektoninstallerset/client/cleanup.go 61.4% 46.6% -14.8
pkg/reconciler/kubernetes/tektoninstallerset/client/create.go 77.6% 69.0% -8.6
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@jkhelil jkhelil force-pushed the statefulset_ordinals branch 2 times, most recently from 2b70d2f to edec38a Compare October 1, 2024 12:43
@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 90.1% -3.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.0% 1.8
pkg/reconciler/common/transformers.go 74.4% 70.4% -4.0
pkg/reconciler/kubernetes/tektoninstallerset/client/cleanup.go 61.4% 46.6% -14.8
pkg/reconciler/kubernetes/tektoninstallerset/client/create.go 77.6% 69.0% -8.6
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@jkhelil jkhelil force-pushed the statefulset_ordinals branch 2 times, most recently from fd9970c to aefcc64 Compare October 1, 2024 12:51
@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 90.1% -3.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.0% 1.8
pkg/reconciler/common/transformers.go 74.4% 70.4% -4.0
pkg/reconciler/kubernetes/tektoninstallerset/client/cleanup.go 61.4% 46.6% -14.8
pkg/reconciler/kubernetes/tektoninstallerset/client/create.go 77.6% 69.0% -8.6
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@tekton-robot
Copy link
Contributor

The following is the coverage report on the affected files.
Say /test pull-tekton-operator-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/operator/v1alpha1/tektonpipeline_defaults.go 93.3% 90.1% -3.2
pkg/apis/operator/v1alpha1/tektonpipeline_validation.go 84.2% 86.4% 2.2
pkg/reconciler/common/transformers.go 74.4% 70.4% -4.0
pkg/reconciler/kubernetes/tektoninstallerset/client/cleanup.go 61.4% 46.6% -14.8
pkg/reconciler/kubernetes/tektoninstallerset/client/create.go 77.6% 69.0% -8.6
pkg/reconciler/kubernetes/tektonpipeline/transform.go 89.8% 89.1% -0.7

@tekton-robot
Copy link
Contributor

@jkhelil: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-tekton-operator-build-tests ec55233 link true /test pull-tekton-operator-build-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@piyush-garg piyush-garg Awaiting requested review from piyush-garg

@PuneetPunamiya PuneetPunamiya Awaiting requested review from PuneetPunamiya

Assignees
No one assigned
Labels
release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jkhelil @tekton-robot