Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abris not working when used with confluent schema registry over basic auth (found on dataproc) #364

Open
DwivediPrasoon opened this issue Sep 17, 2024 · 1 comment

Comments

@DwivediPrasoon
Copy link

DwivediPrasoon commented Sep 17, 2024

This is an issue with confluent-schema-registry-client though but getting extended to Abris. The issue is, when schema registry is being used with basic auth over a confluent instance, it is showing 401 while fetching schema from confluent (incase of dataproc cluster over a private VPN). One can not just pass credentials like following:

AbrisConfig.fromConfluentAvro.downloadReaderSchemaByLatestVersion
          .andTopicNameStrategy(getTopic)
          .usingSchemaRegistry(
            Map(
              AbrisConfig.SCHEMA_REGISTRY_URL -> Configuration.schemaRegistryUrl,
              SchemaRegistryClientConfig.BASIC_AUTH_CREDENTIALS_SOURCE -> "USER_INFO",
              SchemaRegistryClientConfig.USER_INFO_CONFIG -> s"${Configuration.schemaRegistryUsername}:${Configuration.schemaRegistryPassword}"
            )
          )

instead confluent can give you schema if used with a rest service created and passed through. e.g.

val restService = new RestService(urls)
    val provider = new UserInfoCredentialProvider()
    provider.configure(configs.asJava)
    restService.setBasicAuthCredentialProvider(provider)
    new CachedSchemaRegistryClient(restService, maxSchemaObject)

The current implementation of this is not allowing to use restService based constructor with CachedSchemaRegistryClient, so I have tried changing Abris code and made a separate jar with above change. Raising a PR to add this feature.

@DwivediPrasoon
Copy link
Author

https://github.com/AbsaOSS/ABRiS/pull/365 for above issue

@DwivediPrasoon DwivediPrasoon changed the title Abris not working in dataproc when used with confluent schema registry Abris not working in dataproc when used with confluent schema registry over basic auth Sep 17, 2024
@DwivediPrasoon DwivediPrasoon changed the title Abris not working in dataproc when used with confluent schema registry over basic auth Abris not working when used with confluent schema registry over basic auth (found on dataproc) Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant