FileStreamSinkLog

FileStreamSinkLog is a concrete CompactibleFileStreamLog (of SinkFileStatuses) for FileStreamSink and MetadataLogFileIndex.

FileStreamSinkLog uses 1 for the version.

FileStreamSinkLog uses add action to create new metadata logs.

FileStreamSinkLog uses delete action to mark metadata logs that should be excluded from compaction.

Creating FileStreamSinkLog Instance

FileStreamSinkLog (like the parent CompactibleFileStreamLog) takes the following to be created:

  • Metadata version

  • SparkSession

  • Path of the metadata log directory

compactLogs Method

compactLogs(logs: Seq[SinkFileStatus]): Seq[SinkFileStatus]
Note
 compactLogs is part of the CompactibleFileStreamLog Contract to…​FIXME.

compactLogs…​FIXME

results matching ""

    No results matching ""