AppendData Logical Operator — Appending Data to DataSourceV2

AppendData is a logical operator that represents appending data (the result of executing a structured query) to a table (with the columns matching by name or position) in Data Source API V2.

AppendData is created (indirectly via byName or byPosition factory methods) only for tests.

Note
AppendData has replaced the deprecated WriteToDataSourceV2 logical operator.

AppendData takes the following to be created:

  • NamedRelation for the table (to append data to)

  • Logical operator (for the query)

  • isByName flag

AppendData has a single child logical operator that is exactly the logical operator.

AppendData is resolved using ResolveOutputRelation logical resolution rule.

AppendData is planned (replaced) to WriteToDataSourceV2Exec physical operator (when the table is a DataSourceV2Relation logical operator).

byName Factory Method

byName(table: NamedRelation, df: LogicalPlan): AppendData

byName simply creates a AppendData logical operator with the isByName flag on (true).

Note
byName seems used only for tests.

byPosition Factory Method

byPosition(table: NamedRelation, query: LogicalPlan): AppendData

byPosition simply creates a AppendData logical operator with the isByName flag off (false).

Note
byPosition seems used only for tests.

results matching ""

    No results matching ""