Kafka Schema Register with curl

Question

Command used to register schema:

curl -i -XPOST -u "accesskey:secretkey" -H "Content-Type: application/vnd.schemaregistry.v1+json" https://schema-registry.nonprod.us-west-2.aws.proton.x.com/subjects/tracking-info-publish-avro-value/versions --data '{ "schema": "{""type"":""record"",""name"":""ShipmentNotification"",""namespace"":""com.x.ordertrackinginfo.eventschema"",""fields"":[{""name"":""sellingStoreNumber"",""type"":""int""}}'

Error:

Unexpected character (''' (code 39)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false') at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 2]curl: (6) Could not resolve host: schema curl: (3) [globbing] nested brace in column 115

Answer 1

Note: This answer assumes that you're calling from PowerShell and that you're trying to nest JSON and that your JSON is composed as follows:

• A single, top-level schema property

• whose value is a single string that happens to contain JSON too.

If you're calling from bash (or similar), all you need to do is to replace all "" with \" inside the '...' string and to insert a " before the final }.

tl;dr, as of PowerShell 7.1:

curl -i -XPOST -u "accesskey:secretkey" `
     -H "Content-Type: application/vnd.schemaregistry.v1+json" `
     https://schema-registry.nonprod.us-west-2.aws.proton.x.com/subjects/tracking-info-publish-avro-value/versions `
     --data (@'
      { "schema": "{\"type\":\"record\",\"name\":\"ShipmentNotification\",\"namespace\":\"com.x.ordertrackinginfo.eventschema\",\"fields\":[{\"name\":\"sellingStoreNumber\",\"type\":\"int\"}" }
'@ -replace '([\\]*)"', '$1$1\"')

Note the use of ` at the very end of the lines in order to continue the command on the next line (purely for readability) and that the closing '@ delimiter must be the very first character on the line.

---

Step-by-step instructions:

First, create this JSON string in isolation, using a verbatim here-string in PowerShell (although a regular verbatim string, '...', will do), with " characters that are part of the property value (as opposed to those that have syntactic function) escaped as \", as required by JSON:

$json = @'
{ 
  "schema": "{\"type\":\"record\",\"name\":\"ShipmentNotification\",\"namespace\":\"com.x.ordertrackinginfo.eventschema\",\"fields\":[{\"name\":\"sellingStoreNumber\",\"type\":\"int\"}"
}
'@

The resulting string is valid JSON, as you can verify with $json | ConvertFrom-Json

Next, additional escaping must be applied in order to pass the string to an external program:

• Note:
  • This extra round of escaping should never have been necessary, but is up until at least PowerShell version 7.1 (the current version as of this writing).
  • An experimental feature in preview versions of v7.2 can be configured to fix this, so that this step is no longer necessary.
  • See this answer for background information and details.

$jsonEscapedForExternalProgram = $json -replace '([\\]*)"', '$1$1\"'

Now you can pass the escaped JSON string to the external curl program (curl.exe on Windows):

curl -i -XPOST -u "accesskey:secretkey" `
     -H "Content-Type: application/vnd.schemaregistry.v1+json" `
     https://schema-registry.nonprod.us-west-2.aws.proton.x.com/subjects/tracking-info-publish-avro-value/versions `
     --data $jsonEscapedForExternalProgram

Answer 2

I was having the same problem. My solution was to just use postman with the proper headers and sending a json with just the schema attribute as is explained here: https://docs.confluent.io/platform/current/schema-registry/develop/api.html#schemaregistry-api

You can use a json like this:

{ "schema": "{ \"type\": \"record\", \"name\": \"test\", \"fields\": [ { \"type\": \"string\", \"name\": \"field1\" }, { \"type\": \"int\", \"name\": \"field2\" } ] }" }

That json was taken from the documentation.