Skip to content

resolves #1102 : add full primary key support to clickhouse DBMS#1131

Open
abdala-elgendy wants to merge 45 commits into
sqlancer:mainfrom
abdala-elgendy:main
Open

resolves #1102 : add full primary key support to clickhouse DBMS#1131
abdala-elgendy wants to merge 45 commits into
sqlancer:mainfrom
abdala-elgendy:main

Conversation

@abdala-elgendy

@abdala-elgendy abdala-elgendy commented Mar 15, 2025

Copy link
Copy Markdown

resolves #1102
after I traverse the clickhouse and know the flow of creating I see it will be helpful to to add pk to columns
there is only one problem I can't handle well testClickHouseNoREC to deal with NULL values and primary key properly how can I do that dr @qoega could you take a look in the code and this error
image
I handle it in TLP but I can't handle it in NoREC

abdala-elgendy and others added 30 commits March 13, 2025 14:28
…,getPositiveIntegerNotNull,getNonCachedInteger)
This reverts commit 96065ad.
…gDecimal,getPositiveIntegerNotNull,getNonCachedInteger)"

This reverts commit d46dcae.
@abdala-elgendy

Copy link
Copy Markdown
Author

I am sorry about the unrelated log commits but all the changes related to adding a primary key to clickhouse @mrigger

@abdala-elgendy

Copy link
Copy Markdown
Author

Now clickhouse support primary keys could you check dr @qoega

import sqlancer.clickhouse.ClickHouseProvider.ClickHouseGlobalState;
import sqlancer.clickhouse.ClickHouseVisitor;

public class ClickHouseTLPWhereOracle extends ClickHouseTLPBase {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need a separate ClickHouse oracle class? @malwaregarry recently eliminated redundancies. Also, how does this relate to the primary key constraints?

@abdala-elgendy abdala-elgendy Mar 16, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does this relate to the primary key constraints?

No, (clickhouseTLPBase )
handles already all pk constraints I have put in clickhouseTableGenerator
I can delete it with no conflicts in any constraints

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I understand your response. It seems the change is not related to the primary key support? Please only open PRs that have a common objective.

@abdala-elgendy abdala-elgendy Mar 16, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added ClickHouseTLPWhereOracle because there is an error dealing with join_with_nulls and aggregate_functions_null while adding primary keys
image
I couldn't handle these constraints in base class. could you tell me how to handle I will delete ClickHouseTLPWhereOracle.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you perhaps explain why the test oracles fail without the change based on a minimal SQL example? There seems to be a deeper issue here.

@abdala-elgendy abdala-elgendy Mar 22, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry I have seen this comment now dr @mrigger!
after I added pk support the ClickHouse's join behavior changed and without join_use_nulls=1 , NULL handling in joins become inconsistent
image

There seems to be a deeper issue here.

there isn't a deeper issue it is just configuring these settings appropriately, you can achieve more constant query results in Clickhouse.
image

CERTOracle.CheckedFunction<SQLancerResultSet, Optional<Long>> rowCountParser = (rs) -> {
String content = rs.getString(2);
return Optional.of((long) Double.parseDouble(content));
try {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please ensure to not include unrelated changes.

return values;
}

@Test

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems unrelated to ClickHouse.

@abdala-elgendy abdala-elgendy requested a review from mrigger March 20, 2025 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Primary Key to Support for MergeTree in TLP Testing in clickhouse

2 participants