O3-310: allow Put operations on an application-writable config.json file to openmrs/data/frontends/config.json #595

jnsereko · 2024-01-03T10:17:48Z

Description of what I changed

This is a continuation of openmrs/openmrs-module-spa#57 cc @dkayiwa

Issue I worked on

see https://issues.openmrs.org/browse/O3-310

Checklist: I completed these to help reviewers :)

My IDE is configured to follow the code style of this project.

No? Unsure? -> configure your IDE, format the code and add the changes with git add . && git commit --amend
I have added tests to cover my changes. (If you refactored
existing code that was well tested you do not have to add tests)

No? -> write tests and add them to this commit git add . && git commit --amend
I ran mvn clean package right before creating this pull request and
added all formatting changes to my commit.

No? -> execute above command
All new and existing tests passed.

No? -> figure out why and add the fix to your commit. It is your responsibility to make sure your code works.
My pull request is based on the latest changes of the master branch.

No? Unsure? -> execute command git pull --rebase upstream master

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java

dkayiwa · 2024-01-03T20:48:10Z

@jnsereko shouldn't we also have the option to locally download the config.json file from the server storage via a GET HTTP request?

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java

ibacher · 2024-01-04T15:36:37Z

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java

+        }
+    }
+
+    private boolean isValidAuthFormat(HttpServletResponse response, String basicAuth) {


Why do we even have an authorization check here? This can be handled by the filter in this module and we should be able to just reject things if there's no user or the user isn't an administrator.

I have some doubts regarding our current authorisation filters.
When i execute a request with/without an Auth header, it fails the first time but succeeds on subsequent tries, hence i resorted to retaining the authorisation confirmation logic. However, i am going to remove it.

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java

ibacher · 2024-01-04T15:38:46Z

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java

+        if (!folder.isDirectory()) {
+            log.warn("SPA frontend repository is not a directory hence creating it at: " + folder.getAbsolutePath());
+            if (!folder.mkdirs()) {
+                throw new ModuleException("Failed to create the SPA frontend repository folder at: " + folder.getAbsolutePath());


Why ModuleException? That shouldn't be used here. Also, it's not great to echo a file path back in response to a web request...

I think throwing an IOException is better

dkayiwa · 2024-01-10T12:30:22Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+
+    private static final Logger log = LoggerFactory.getLogger(FrontendJsonConfigController1_9.class);
+
+    @RequestMapping(method = RequestMethod.GET)


The formatting here looks ... 😊

Did you address the formatting?

I have removed the extra spaces

And the indention?

There should be a `config.json` file that the application is allowed to manage, which lives in the application data directory.

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

dkayiwa · 2024-01-11T11:24:04Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+            BufferedReader reader = request.getReader();
+            StringBuilder stringBuilder = new StringBuilder();
+            String line;
+            while ((line = reader.readLine()) != null) {


Don't we have a library like IOUtils or any other to read this?

And are we doing all this simply for the sake of validating that it is a JSON file?

new ObjectMapper().readTree(requestBody);
this is the only line that validates the JSON file. Is it so expensive?

Then why is the entire method block surrounded with a try catch for JsonProcessingException?

Then why is the entire method block surrounded with a try catch for JsonProcessingException?

I've made the try block specific to parsing

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

dkayiwa · 2024-01-11T22:43:42Z

@jnsereko can you respond to all my comments?

jnsereko · 2024-01-12T05:31:23Z

@jnsereko can you respond to all my comments?

Sorry i had missed some of them. This PR has quite many comments. I hope i have addressed all of them now

dkayiwa · 2024-01-12T11:58:01Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+    private void saveJsonConfigFile(HttpServletRequest request, HttpServletResponse response) throws IOException {
+        File jsonConfigFile = getJsonConfigFile();
+        InputStream inputStream = request.getInputStream();
+        String requestBody = IOUtils.toString( inputStream , "UTF-8");


Is the above line also for validation?

no, this is just for reading the request body. Line 86 does the validation

ibacher · 2024-01-12T14:01:30Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+    private static final Logger log = LoggerFactory.getLogger(FrontendJsonConfigController1_9.class);
+
+    @RequestMapping(method = RequestMethod.GET)
+	public void getFrontendConfigFile(HttpServletRequest request, HttpServletResponse response) throws IOException {


There's some inconsistency in spacing here. This line uses a tab character, but most lines are indented by spaces.

i am wondering, what brought that tab, locally it doesn't exist. I will re-push

IDEs have a nasty habit of doing this on auto-indent.

ibacher · 2024-01-12T14:01:45Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+            log.error("Error reading Configuration file: " + jsonConfigFile.getAbsolutePath(), e);
+            response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Error reading Configuration file: " + jsonConfigFile.getPath());
+        }
+	}


This is the other line I see indented by a tab.

same as here #595 (comment)

ibacher · 2024-01-12T14:02:09Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+            response.setHeader("Content-Disposition", "attachment; filename=" + jsonConfigFile.getName());
+            response.setStatus(HttpServletResponse.SC_OK);
+        } catch (IOException e) {
+            log.error("Error reading Configuration file: " + jsonConfigFile.getAbsolutePath(), e);


Suggested change

log.error("Error reading Configuration file: " + jsonConfigFile.getAbsolutePath(), e);

log.error("Error reading Configuration file {}", jsonConfigFile.getAbsolutePath(), e);

ibacher · 2024-01-12T14:05:18Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+            OpenmrsUtil.copyFile(inputStream, outStream);
+
+            response.setContentType("application/json");
+            response.setHeader("Content-Disposition", "attachment; filename=" + jsonConfigFile.getName());


Adding Content-Disposition: attachment makes it impossible to use this on the frontend because you're not returning the JSON file as a REST response. You're returning a response instructing the user agent to download that as a file. Without some kind of endpoint that allows us to make a get request and get the JSON document back as part of the response without the Content-Disposition we cannot make use of this configuration file. (I'd suggest adding an optional parameter, e.g., ?download so that:

GET /openmrs/ws/rest/v1/frontend/config.json returns like requesting any other JSON file and GET /openmrs/ws/rest/v1/frontend/config.json?download sends it as an attachment.

i have added the Content-Disposition: attachment for both GET /openmrs/ws/rest/v1/frontend/config.json?download and GET /openmrs/ws/rest/v1/frontend/config.json?download=true. As an addition, i have Set their content type to application/x-download
Thank you for this clarification

I'd keep the content type! It's still a JSON document.

ibacher · 2024-01-12T14:05:40Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+	public void getFrontendConfigFile(HttpServletRequest request, HttpServletResponse response) throws IOException {
+        File jsonConfigFile = getJsonConfigFile();
+        if (!jsonConfigFile.exists()) {
+            log.debug("Configuration file does not exist");


Suggested change

log.debug("Configuration file does not exist");

log.warn("Configuration file does not exist");

ibacher · 2024-01-12T14:10:12Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+            // verify that is in a valid JSON format
+            new ObjectMapper().readTree(requestBody);
+        } catch (JsonProcessingException e) {
+            log.error("Invalid JSON format", e);


Suggested change

log.error("Invalid JSON format", e);

log.error("Invalid JSON format when reading: {}", requestBody, e);

ibacher · 2024-01-12T14:11:34Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+        OutputStream outStream = Files.newOutputStream(jsonConfigFile.toPath());
+        OpenmrsUtil.copyFile(inputStream, outStream);
+
+        if (jsonConfigFile.exists()) {


This check is unnecessary. copyFile() would've thrown an exception (which should potentially be handled here) if there was a failure.

ibacher · 2024-01-12T14:11:55Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+        try {
+            // verify that is in a valid JSON format
+            new ObjectMapper().readTree(requestBody);
+        } catch (JsonProcessingException e) {
+            log.error("Invalid JSON format", e);
+            response.sendError(HttpServletResponse.SC_BAD_REQUEST, "Invalid JSON format");
+        }


Suggested change

try {

// verify that is in a valid JSON format

new ObjectMapper().readTree(requestBody);

} catch (JsonProcessingException e) {

log.error("Invalid JSON format", e);

response.sendError(HttpServletResponse.SC_BAD_REQUEST, "Invalid JSON format");

}

try {

// verify that is in a valid JSON format

new ObjectMapper().readTree(requestBody);

} catch (JsonProcessingException e) {

log.error("Invalid JSON format", e);

response.sendError(HttpServletResponse.SC_BAD_REQUEST, "Invalid JSON format");

}

ibacher · 2024-01-12T14:13:38Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+        if (jsonConfigFile.exists()) {
+            log.debug("file: '{}' written successfully", jsonConfigFile.getAbsolutePath());
+            response.setStatus(HttpServletResponse.SC_OK);
+        }


Never end a function like this with an if without an else. What response would the consumer get if the jsonConfigFile.exists() check returned false? (This is just intended as a pedagogic remark; as I said above, the entire if is unnecessary

ibacher · 2024-01-12T14:14:57Z

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java

+        if (!folder.isDirectory()) {
+            log.debug("Unable to find the OpenMRS SPA module frontend directory hence creating it at: " + folder.getAbsolutePath());
+            if (!folder.mkdirs()) {
+                throw new IOException("Failed to create the OpenMRS SPA module frontend directory at: " + folder.getPath());


This is fine, but:

The error should be logged

The consuming code should probably catch this exception and return something without the full path

ibacher · 2024-01-16T21:59:34Z

It'd be great to have some tests for this.

brandones · 2024-06-04T19:16:07Z

@jnsereko Do you think you could add some tests that verify that this works as expected?

jnsereko mentioned this pull request Jan 3, 2024

O3-310: allow Put operations on an application-writable config.json file to openmrs/data/frontends/config.json openmrs/openmrs-module-spa#57

Closed

dkayiwa reviewed Jan 3, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java Outdated Show resolved Hide resolved

dkayiwa reviewed Jan 3, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java Outdated Show resolved Hide resolved

dkayiwa reviewed Jan 3, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java Outdated Show resolved Hide resolved

dkayiwa reviewed Jan 3, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java Outdated Show resolved Hide resolved

dkayiwa reviewed Jan 3, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java Outdated Show resolved Hide resolved

dkayiwa reviewed Jan 3, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java Outdated Show resolved Hide resolved

dkayiwa reviewed Jan 3, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java Outdated Show resolved Hide resolved

dkayiwa reviewed Jan 3, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java Outdated Show resolved Hide resolved

ibacher reviewed Jan 4, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs2_4/FrontendJsonConfigController2_4.java Outdated Show resolved Hide resolved

ibacher reviewed Jan 4, 2024

View reviewed changes

jnsereko requested review from dkayiwa and ibacher January 8, 2024 11:19

dkayiwa reviewed Jan 10, 2024

View reviewed changes

O3-310: allow Put operations on an application-writable config.json file

a44a9d3

There should be a `config.json` file that the application is allowed to manage, which lives in the application data directory.

jnsereko force-pushed the O3-310 branch from e5818ad to a44a9d3 Compare January 10, 2024 14:32

dkayiwa reviewed Jan 11, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java Outdated Show resolved Hide resolved

dkayiwa reviewed Jan 11, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java Outdated Show resolved Hide resolved

dkayiwa reviewed Jan 11, 2024

View reviewed changes

.../module/webservices/rest/web/v1_0/controller/openmrs1_9/FrontendJsonConfigController1_9.java Outdated Show resolved Hide resolved

ibacher mentioned this pull request Jan 11, 2024

feat: O3-310: Add an application-writable config.json file in frontends/ (attempt #2) openmrs/openmrs-esm-core#865

Open

use IOUtils for oneline line reading

9c6d3ea

jnsereko requested a review from dkayiwa January 11, 2024 17:42

jnsereko added 2 commits January 12, 2024 08:17

makee the auth error message more helpful

1abae4c

remove unrequired spaces

bf46700

move json validation to a loner try

6ffb3dd

dkayiwa reviewed Jan 12, 2024

View reviewed changes

ibacher reviewed Jan 12, 2024

View reviewed changes

jnsereko added 2 commits January 15, 2024 16:04

add support for the download parameter

a78f76d

fix formarting issue

b426de6

jnsereko requested review from dkayiwa and ibacher January 15, 2024 13:28

jnsereko and others added 2 commits January 16, 2024 01:03

reveret header content type to JSON

b6ef2eb

remove empty spaces

3abd011


		private static final Logger log = LoggerFactory.getLogger(FrontendJsonConfigController1_9.class);

		@RequestMapping(method = RequestMethod.GET)

	log.error("Error reading Configuration file: " + jsonConfigFile.getAbsolutePath(), e);
	log.error("Error reading Configuration file {}", jsonConfigFile.getAbsolutePath(), e);

	log.debug("Configuration file does not exist");
	log.warn("Configuration file does not exist");

	log.error("Invalid JSON format", e);
	log.error("Invalid JSON format when reading: {}", requestBody, e);

O3-310: allow Put operations on an application-writable config.json file to openmrs/data/frontends/config.json #595

Are you sure you want to change the base?

O3-310: allow Put operations on an application-writable config.json file to openmrs/data/frontends/config.json #595

Conversation

jnsereko commented Jan 3, 2024 • edited Loading

Description of what I changed

Issue I worked on

Checklist: I completed these to help reviewers :)

dkayiwa commented Jan 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dkayiwa Jan 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnsereko Jan 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dkayiwa commented Jan 11, 2024

jnsereko commented Jan 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ibacher commented Jan 16, 2024

brandones commented Jun 4, 2024

jnsereko commented Jan 3, 2024 •

edited

Loading

dkayiwa commented Jan 3, 2024 •

edited

Loading

dkayiwa Jan 10, 2024 •

edited

Loading

jnsereko Jan 12, 2024 •

edited

Loading