Performance drop from 1.x to 3.x

[rdominy]

We use this in a log processing tool and after upgrading from 1.x to 3.x we saw a drop of 25% in our log processing throughput. After directly perf testing 1.x vs 3.x, v3.x is about 2 times slower than v1.x. While most apps are unlikely to even notice a drop, anything calling this at scale is likely to see a performance drop.

[mrodrig]

Hi @rdominy. Thanks for reporting this and my apologies for the delay in getting back to you. Do you happen to have a code snippet/sample data that could be used to performance test the different versions of the module? I'd be happy to take a look at the various code changes to see what may be negatively impacting it. My suspicion is that some of the performance degradation may be due to additional logic that was necessary to mitigate a possible prototype pollution vulnerability, but it's also possible that it could be due to other changes. Thanks again!

[mrodrig]

Any code snippets or sample data you can provide would be greatly appreciated. I setup a test script which compares doc-path@1.3.1 versus doc-path@3.0.6 and regardless of which version is executed first, the worst-case performance of 3.0.6 is consistently better than 1.3.1 for every sample I try.

[rdominy]

Here’s the test i used: "use strict"; let CONFIG = { "splunk": { "token": "FFD1D338-12AB-4B38-899-ZZZZZ4AAA", "url": "https://splunk.sampleurl..net:8088" }, "name": "This is a thingy", "filter": "FooBar stats", "columns": ["epoch", "msg", "data"], "conversions": { "msg": { "start": "scale=", "type": "querystring", "numbers": true }, "data": { "type": "json" }, "data.url": { "start": "?", "type": "querystring", "numbers": false } }, "groupBy": "data.place", "header": "[now.yyyy]-[now.mm]-[now.dd] [now.hh]:[now.mi]:[now.ss] [data.place]", "output": { "requests_c": ["count"], "bids_c": ["countIf", "data.bid"], "adslot": ["stats", "msg.c_adslot"], "uid_c": ["sum", "msg.c_uid"], "zip_c": ["sum", "msg.c_zip"], "latlon_c": ["sum", "msg.c_latlon"], "dma_c": ["sum", "msg.c_dma"], "dtuid_c": ["sum", "msg.c_dtuid"], "site_c": ["sum", "msg.c_site"], "min_cpm_micros": ["stats", "msg.c_min_cpm_micros"], "elapsed": ["stats", "msg.total"], "price": ["statsIf", "data.bid", "data.price"], "pstrat": ["distinctIf", "data.bid", "data.ps"], "nbr": ["distinct", "data.nbr"], "viewability": ["statsIf", "msg.c_viewability", "msg.c_viewability"], "oracle_c": ["sum", "msg.c_oracle"], "vast2": ["sum", "msg.c_vast2"], "vast3": ["sum", "msg.c_vast3"], "vast4": ["sum", "msg.c_vast4"], "seen": ["distinct", "data.seen_state"], "dealid": ["distinct", "data.dealid"], "deals": ["stats", "msg.c_deals"], "floor": ["statsIf", "data.floor", "data.floor"], "nobudg": ["sum", "msg.c_no_budget"] } }; const ITERATIONS = 1000000; let start = Date.now(); const DocPath=require('doc-path') for (let i=0;i<ITERATIONS;i++) { DocPath.evaluatePath(CONFIG, "spunk.url"); DocPath.evaluatePath(CONFIG, "output.floor"); DocPath.evaluatePath(CONFIG, "output.foobar"); DocPath.evaluatePath(CONFIG, "conversions.msg.start"); } console.log("getPath", Date.now()-start); Times I see locally are: v3: 919ms v1: 418ms I ended up replacing the code for this particular use case because it was in a high scale loop, so it could be that this issue won’t affect most users.

…

On Oct 10, 2022, at 9:13 PM, Michael Rodrigues ***@***.***> wrote: Any code snippets or sample data you can provide would be greatly appreciated. I setup a test script which compares ***@***.*** versus ***@***.*** and regardless of which version is executed first, the worst-case performance of 3.0.6 is consistently better than 1.3.1 for every sample I try. — Reply to this email directly, view it on GitHub <#27 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJT45Y5ZOX3NQR45BD2Y3TWCTSQDANCNFSM5UQYGMBA>. You are receiving this because you were mentioned.

[mrodrig]

Sorry for the delay and thanks for the test code @rdominy. I just checked on a Raspberry Pi 4, so a less than ideal machine, and can confirm that I am seeing a difference between the v1 and v4 releases:

v4:

getPath 12534

v1:

getPath 3392

Sorry to hear you had to swap out the code. Even though it won't help at this point, I'm also curious to see what change(s) are ultimately responsible for this so I'm still planning to look into this and possible optimizations to help restore the original v1 release's speed.