feat: add OpenSearch distance operations and selectors by mmiguerodriguez · Pull Request #1335 · WebFuzzing/EvoMaster

mmiguerodriguez · 2025-10-02T13:17:04Z

No description provided.

…-operations

jgaleotti

please add test cases for the classes OpenSearchHeuristicsCalculator, ParameterExtractor and OpenSearchQueryHelper

jgaleotti · 2025-10-29T18:26:47Z

...ler/src/main/java/org/evomaster/client/java/controller/opensearch/OpenSearchQueryParser.java

+import org.evomaster.client.java.controller.opensearch.selectors.MatchSelector;
 import org.evomaster.client.java.controller.opensearch.selectors.QuerySelector;

 public class OpenSearchQueryParser {


We need tests for OpenSearchQueryParser to check that the JSON is correctly parsed into its corresponding OpenSearch operation.

.../src/main/java/org/evomaster/client/java/controller/opensearch/utils/ParameterExtractor.java

...c/main/java/org/evomaster/client/java/controller/opensearch/utils/OpenSearchQueryHelper.java

…-operations

arcuri82

sorry for late check, but was on vacation most of last week

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

arcuri82 · 2025-11-18T12:14:39Z

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

+
+        try {
+            // Create pattern with case insensitive flag if needed
+            java.util.regex.Pattern pattern;


are the regex in OpenSearch having the same syntax of Java regex? clarify here in some comments, with possible some links

OpenSearch uses Lucene regular expression syntax, which is similar but NOT identical to Java regex.

https://opensearch.org/docs/latest/query-dsl/term/regexp/
https://lucene.apache.org/core/8_0_0/core/org/apache/lucene/util/automaton/RegExp.html

@jgaleotti Do you consider adding Java Regexp for this heuristic calculation as acceptable?

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

...-tests/spring/spring-rest-opensearch/src/main/java/com/opensearch/age/OpenSearchAgeRest.java

arcuri82 · 2025-11-18T12:25:36Z

...st/java/org/evomaster/e2etests/spring/rest/opensearch/students/OpenSearchStudentsEMTest.java

                Solution<RestIndividual> solution = initAndRun(args);

-//                assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/students/{q}", null);
+                assertHasAtLeastOne(solution, HttpVerb.GET, 200, "/students/{lastName}", null);


add assertions for both 404 and 200 cases

arcuri82 · 2025-11-18T12:27:13Z

...tests/spring-rest-opensearch/src/main/java/com/opensearch/queries/OpenSearchQueriesRest.java

+    // Term selector tests
+    @GetMapping("category/{category}")
+    public List<Product> findByCategory(@PathVariable String category) throws IOException {
+        return productRepository.findByCategory(category);


do return 404 if nothing is found in these endpoints, as it will simplify writing assertions/checks in the tests

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java

arcuri82 · 2025-11-18T12:31:58Z

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java

+    }
+
+    @Test
+    public void testRangeQueries() throws Throwable {


why are these E2E repeated with same configurations? can have single tests with all the assertions.
otherwise, if you only want to test specific endpoints, you have to pass args parameters to specify them, eg, with --endpointPrefix

...spring/spring-rest-opensearch/src/main/java/com/opensearch/queries/OpenSearchQueriesApp.java

…-operations

mmiguerodriguez · 2025-11-18T23:26:07Z

@arcuri82 is there any known issue with the evomaster-e2e-tests-spring-rest-rsa module? Most of the times I try building the project I'm having issues with that...

[INFO] evomaster-e2e-tests-spring-rest-rsa ................ FAILURE [  0.150 s]
[INFO] evomaster-e2e-tests-spring-rpc-grpc ................ SKIPPED
[INFO] evomaster-e2e-tests-spring-rpc-thrift .............. SKIPPED
[INFO] evomaster-core-integration-tests ................... SKIPPED
[INFO] evomaster-core-driver-it ........................... SKIPPED
[INFO] evomaster-core-graphql-it .......................... SKIPPED
[INFO] evomaster-core-it .................................. SKIPPED
[INFO] evomaster-test-old-libraries ....................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

Is there any way to prevent those errors?

arcuri82 · 2025-11-20T12:14:44Z

@mmiguerodriguez hi. there is no known current issue about evomaster-e2e-tests-spring-rest-rsa. what is the error you are getting?

mmiguerodriguez · 2025-11-20T13:11:46Z

I'm not completely sure what was the issue, perhaps different Java version used by Maven and IntelliJ builds...
I've modified my IntelliJ to use Java 17 and maven too. Now it's fixed.

I'll be reviewing why there's some errors on the build/tests. I've already made some of the adjustements.

arcuri82 · 2025-11-20T13:13:11Z

@mmiguerodriguez ah, yes. we moved to JDK 17 recently... and might move to 21 in the near future

mmiguerodriguez · 2025-11-20T13:13:56Z

We'll need to adjust for_developers.md since it may be outdated.

arcuri82 · 2025-11-20T13:17:37Z

@mmiguerodriguez good point! i ll do now

…-operations

jgaleotti · 2026-01-16T15:46:52Z

Hi @mmiguerodriguez
We noticed an issue with SutController.initOpenSearchHandler()
The following code:

        List<AdditionalInfo> list = getAdditionalInfoList();
        if (!list.isEmpty()) {
            AdditionalInfo last = list.get(list.size() - 1);
            last.getOpenSearchInfoData().forEach(openSearchHandler::handle);
        }

is a copy paste error from SutController.initMongoHanlder(). In this method, the call is made to MongoHandler.handle(MongoCollectionSchema), not to MongoHanlder.handle(MongoFindCommand). In other words, this code is used to infer the expected classes of each Mongo collection, not to handle specific queries that were issued during initialization.

mmiguerodriguez · 2026-01-16T16:42:39Z

@jgaleotti

Is this causing issues on the release?

I’ll be reviewing it.

jgaleotti · 2026-01-16T18:14:00Z

No, it is not causing issues in the release, but it would be nice to get this fixed in this PR

…-operations

mmiguerodriguez · 2026-02-03T20:20:42Z

@jgaleotti I think we can review this once more, I've already modified the issue you've mentioned, along with the original motivation of this PR.

There's tests which are commented since we still don't have OpenSearch queries generation yet.

core-tests/e2e-tests/spring/spring-rest-opensearch/pom.xml

…-operations

arcuri82 · 2026-02-19T13:02:20Z

...roller/src/main/java/org/evomaster/client/java/controller/internal/db/OpenSearchHandler.java

+     * Info about schemas of the documents of the index extracted from Spring framework.
+     * Documents of the index will be mapped to the Repository type
+     */
+     private final Map<String, String> indexSchemas;


as for any map, explicitely state here in the comments what the keys are, and what the values are

I'm not sure about this, since the indexSchema represents the schema of an OpenSearchIndex, we aren't in control of the keys for this schema.

Still I'll be reviewing if there's a better way to map this attribute.

@mmiguerodriguez , maybe it is a misunderstanding, but I just meant to add docuemtnation, eg, something like:

* Key -> <some description> * Value -> <some description> */

in other parts of the code, we have comments like:

/** * key -> malformed URI, found in for example in <a> links * value -> set of urls of pages in which some malformed link was present */ private val malformedLinks : MutableMap<String,MutableSet<String>> = mutableMapOf()

recall what written in for_developers.md

arcuri82 · 2026-02-19T13:21:52Z

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

+            }
+
+            // For non-matches, return a distance based on string similarity
+            return (double) DistanceHelper.getLeftAlignmentDistance(actualStr, regex);


why comparing the alignment of the actualStr with the regex definition? that seems wrong. we have function to compute distances for regex, but shouldn't be used any more (long story). here just return some value above 0.
the handling of regex is better done with taint analysis.
but that can be done in a separated PR.
@jgaleotti you can then explain it to @mmiguerodriguez (you can also see how string equalities and regex are handled in SQL distances to get an idea)

Will review Taint analysis

hi @mmiguerodriguez , let's discuss this.

arcuri82 · 2026-02-19T13:22:34Z

...ain/java/org/evomaster/client/java/controller/opensearch/OpenSearchHeuristicsCalculator.java

+                    return Double.MAX_VALUE; // If any must_not clause matches
+                }
+                // For must_not, closer matches are worse, so we invert the distance logic
+                totalDistance += Math.max(0, 10.0 - distance);


where is the number 10.0 coming from??? unclear

Will review this logic with some tests

arcuri82 · 2026-02-19T13:25:10Z

...e-heuristics/src/main/java/org/evomaster/client/java/distance/heuristics/DistanceHelper.java


+    /**
+     * Calculate edit distance between two strings.
+     */


add comment to explain what allowTranspositions is

arcuri82 · 2026-02-19T13:28:54Z

...e-heuristics/src/main/java/org/evomaster/client/java/distance/heuristics/DistanceHelper.java

+    /**
+     * Calculate standard Levenshtein distance between two strings.
+     */
+    public static int calculateLevenshteinDistance(String s1, String s2) {


why re-implementing this? also, did you copy this code from somewhere? or did you implement it by yourself?
in core (but not in driver), we use org.apache.commons.text.similarity.LevenshteinDistance

I've copied an implementation, will replace with the one which already exists 😓

arcuri82 · 2026-02-19T13:43:14Z

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java

+                // TODO: Commented out - failing because EvoMaster generates queries that result in 500 errors instead of 404
+                // Assert 404 responses when data is not found
+                // assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/queries/price-range", null);
+                // assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/queries/rating-gte/{rating}", null);


instead of commenting out, would be better to mark the test as @Disabled, with a comment to explain why. still, this should be solved in next PR

Will mark as Disabled instead of commenting

arcuri82 · 2026-02-19T13:43:28Z

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java

+                // TODO: Commented out - failing because EvoMaster generates queries that result in 500 errors instead of 404
+                // Assert 404 responses when data is not found
+                // assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/queries/name-prefix/{prefix}", null);
+                // assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/queries/name-fuzzy/{name}", null);


see previous comments

arcuri82 · 2026-02-19T13:43:34Z

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java

+                // TODO: Commented out - failing because EvoMaster generates queries that result in 500 errors instead of 404
+                // Assert 404 responses when data is not found
+                // assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/queries/with-email", null);
+                // assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/queries/complex", null);


see previous comments

arcuri82 · 2026-02-19T13:43:55Z

...test/java/org/evomaster/e2etests/spring/rest/opensearch/queries/OpenSearchQueriesEMTest.java

+                    .count();
+
+                assertTrue(termQueries > 0 || rangeQueries > 0 || textQueries > 0,
+                    "Should have executed some OpenSearch queries");*/


see previous comments

arcuri82 · 2026-02-19T13:44:39Z

...st/java/org/evomaster/e2etests/spring/rest/opensearch/students/OpenSearchStudentsEMTest.java

-//                assertHasAtLeastOne(solution, HttpVerb.GET, 404, "/students/{q}", null);
+                // TODO: Commented out - failing because EvoMaster cannot find valid query parameters
+                // Assert 200 responses when data can be generated by EvoMaster
+                // assertHasAtLeastOne(solution, HttpVerb.GET, 200, "/students/{lastName}", null);


see previous comments. likely this will be easy to cover once implemented taint analysis support

mmiguerodriguez added 9 commits September 23, 2025 19:55

feat: add OpenSearch distance operations and selectors

14c53c2

Merge branch 'master' into feature/os-distance-operations

df6ffe6

feature: added new integration test

a356951

Merge remote-tracking branch 'origin/master' into feature/os-distance…

9da8da8

…-operations

remove invalid assertions

54f1eac

fix tests

23b7b20

more test fixes...

b529331

fix test by using default constructor

b461a0f

Merge branch 'master' into feature/os-distance-operations

bec8eb7

mmiguerodriguez marked this pull request as ready for review October 28, 2025 21:52

jgaleotti requested changes Oct 29, 2025

View reviewed changes

mmiguerodriguez marked this pull request as draft October 29, 2025 18:36

mmiguerodriguez marked this pull request as ready for review October 29, 2025 18:36

mmiguerodriguez added 3 commits November 6, 2025 15:23

tests: add PR recommendations

41fae1f

Merge remote-tracking branch 'origin/master' into feature/os-distance…

ae37f6a

…-operations

Merge remote-tracking branch 'origin/master' into feature/os-distance…

44452ff

…-operations

jgaleotti requested a review from arcuri82 November 14, 2025 13:56

arcuri82 requested changes Nov 18, 2025

View reviewed changes

mmiguerodriguez added 10 commits November 18, 2025 10:03

Merge remote-tracking branch 'origin/master' into feature/os-distance…

f47e055

…-operations

refactor: use if/else chain with warn log for unsupported ops

dfe913b

fix: handle empty expectedTerms to prevent division by zero

5b7e4cd

docs: clarify OpenSearch regex vs Java regex syntax differences

152a361

refactor: move generic string helpers to DistanceHelper

b82ab2d

fix: return 404 when no data found in age endpoints

7b612ad

fix: return 404 in students endpoints and add 404 assertions in tests

e17daa6

fix: return 404 in queries endpoints and add 404 assertions

a874b55

refactor: move queries module to new folder structure

51ab79a

style: remove redundant javadoc comments

4bf9147

mmiguerodriguez added 2 commits January 1, 2026 18:20

Merge remote-tracking branch 'origin/master' into feature/os-distance…

9d6a2b0

…-operations

Merge remote-tracking branch 'origin/master' into feature/os-distance…

f42b67e

…-operations

mmiguerodriguez added 5 commits January 21, 2026 20:50

fix: add env vars and comment out tests with errors

fa3216a

fix handle on wrong entity. wip OpenSearchIndexSchema

4541ab7

Merge remote-tracking branch 'origin/master' into feature/os-distance…

a76e8e4

…-operations

fix: add index schema to ExecutionTracer

173886e

Merge branch 'master' into feature/os-distance-operations

b25da6d

mmiguerodriguez requested a review from jgaleotti February 3, 2026 20:19

jgaleotti requested changes Feb 4, 2026

View reviewed changes

core-tests/e2e-tests/spring/spring-rest-opensearch/pom.xml Outdated Show resolved Hide resolved

mmiguerodriguez added 2 commits February 12, 2026 11:19

fix: remove unused pom.xml configuration

4984d0f

Merge remote-tracking branch 'origin/master' into feature/os-distance…

94dde39

…-operations

mmiguerodriguez requested a review from jgaleotti February 12, 2026 14:20

jgaleotti approved these changes Feb 12, 2026

View reviewed changes

jgaleotti requested a review from arcuri82 February 12, 2026 17:04

arcuri82 added 2 commits February 13, 2026 11:15

Merge branch 'master' into feature/os-distance-operations

d6e76b8

Merge branch 'master' into feature/os-distance-operations

e77f17a

arcuri82 requested changes Feb 19, 2026

View reviewed changes

Comments

Conversation

mmiguerodriguez commented Oct 2, 2025

Uh oh!

jgaleotti left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

arcuri82 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmiguerodriguez Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mmiguerodriguez commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arcuri82 commented Nov 20, 2025

Uh oh!

mmiguerodriguez commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arcuri82 commented Nov 20, 2025

Uh oh!

mmiguerodriguez commented Nov 20, 2025

Uh oh!

arcuri82 commented Nov 20, 2025

Uh oh!

jgaleotti commented Jan 16, 2026

Uh oh!

mmiguerodriguez commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jgaleotti commented Jan 16, 2026

Uh oh!

mmiguerodriguez commented Feb 3, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmiguerodriguez Nov 18, 2025 •

edited

Loading

mmiguerodriguez commented Nov 18, 2025 •

edited

Loading

mmiguerodriguez commented Nov 20, 2025 •

edited

Loading

mmiguerodriguez commented Jan 16, 2026 •

edited

Loading