forked from DTStack/chunjun
-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
bugSomething isn't workingSomething isn't working
Description
illegal where criteria build WHERE "id" > ? ORDER BY "id" ASC AND "id" > ? ORDER BY "id" ASC,
when process logic of PostgresqlInputFormat ,build sql in
Lines 55 to 62 in 2251bcf
| if (jdbcConf.getQuerySql().contains("WHERE")) { | |
| builder.append(" AND "); | |
| } else { | |
| builder.append(" WHERE "); | |
| } | |
| builder.append(jdbcDialect.quoteIdentifier(jdbcConf.getIncreColumn())) | |
| .append(" > ? ORDER BY ") | |
| .append(jdbcDialect.quoteIdentifier(jdbcConf.getIncreColumn())) |
will add duplicate where cause,that's illegal
2022-08-08 14:40:06
com.dtstack.chunjun.throwable.ChunJunRuntimeException: error to execute sql = SELECT "id", "bytea_c", "small_c", "int_c", "big_c", "real_c", "double_precision", "numeric_c", "decimal_c", "boolean_c", "text_c", "char_c", "character_c", "character_varying_c", "timestamp3_c", "timestamp6_c", "date_c", "time_c", "default_numeric_c" FROM "full_types" WHERE "id" > ? ORDER BY "id" ASC AND "id" > ? ORDER BY "id" ASC, startLocation = 1, e = org.postgresql.util.PSQLException: 未设定参数值 2 的内容。
at org.postgresql.core.v3.SimpleParameterList.checkAllParametersSet(SimpleParameterList.java:284)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:338)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:484)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:404)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:162)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:114)
at com.dtstack.chunjun.connector.jdbc.source.JdbcInputFormat.queryForPolling(JdbcInputFormat.java:691)
at com.dtstack.chunjun.connector.jdbc.source.JdbcInputFormat.reachedEnd(JdbcInputFormat.java:259)
at com.dtstack.chunjun.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:137)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:269)
at com.dtstack.chunjun.connector.jdbc.source.JdbcInputFormat.reachedEnd(JdbcInputFormat.java:273)
at com.dtstack.chunjun.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:137)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:269)
Suppressed: java.lang.RuntimeException: com.dtstack.chunjun.throwable.NoRestartException: The dirty consumer shutdown, due to the consumed count exceed the max-consumed [0]
at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.close(BaseRichOutputFormat.java:358)
at com.dtstack.chunjun.sink.DtOutputFormatSinkFunction.close(DtOutputFormatSinkFunction.java:127)
at org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:41)
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.dispose(AbstractUdfStreamOperator.java:117)
at org.apache.flink.streaming.runtime.tasks.StreamTask.disposeAllOperators(StreamTask.java:861)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runAndSuppressThrowable(StreamTask.java:840)
at org.apache.flink.streaming.runtime.tasks.StreamTask.cleanUpInvoke(StreamTask.java:753)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.cleanUpInvoke(SourceStreamTask.java:186)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:659)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:620)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:779)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.dtstack.chunjun.throwable.NoRestartException: The dirty consumer shutdown, due to the consumed count exceed the max-consumed [0]
at com.dtstack.chunjun.dirty.consumer.DirtyDataCollector.addConsumed(DirtyDataCollector.java:105)
at com.dtstack.chunjun.dirty.consumer.DirtyDataCollector.offer(DirtyDataCollector.java:79)
at com.dtstack.chunjun.dirty.manager.DirtyManager.collect(DirtyManager.java:140)
at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeSingleRecord(BaseRichOutputFormat.java:475)
at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.lambda$writeRecordInternal$1(BaseRichOutputFormat.java:493)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeRecordInternal(BaseRichOutputFormat.java:493)
at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.lambda$initTimingSubmitTask$0(BaseRichOutputFormat.java:449)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.postgresql.util.PSQLException: 未设定参数值 2 的内容。
at org.postgresql.core.v3.SimpleParameterList.checkAllParametersSet(SimpleParameterList.java:284)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:338)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:484)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:404)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:162)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:114)
at com.dtstack.chunjun.connector.jdbc.source.JdbcInputFormat.queryForPolling(JdbcInputFormat.java:691)
at com.dtstack.chunjun.connector.jdbc.source.JdbcInputFormat.reachedEnd(JdbcInputFormat.java:259)
... 4 moreMetadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working