FileSystemBasedCheckpointFileManager — CheckpointFileManager on Hadoop’s FileSystem API
FileSystemBasedCheckpointFileManager is a CheckpointFileManager that uses Hadoop’s FileSystem API for managing checkpoint files:
-
mkdirs uses FileSystem.mkdirs
-
createTempFile uses FileSystem.create (with overwrite enabled)
-
createAtomic uses
RenameBasedFSDataOutputStream -
open uses FileSystem.open
-
delete uses FileSystem.delete (with recursive enabled)
-
isLocal is
truefor the FileSystem beingLocalFileSystemorRawLocalFileSystem
FileSystemBasedCheckpointFileManager is created exclusively when CheckpointFileManager helper object is requested for a CheckpointFileManager (for HDFSMetadataLog, StreamMetadata and HDFSBackedStateStoreProvider).
FileSystemBasedCheckpointFileManager is a RenameHelperMethods for atomicity by "write-to-temp-file-and-rename".
Creating FileSystemBasedCheckpointFileManager Instance
FileSystemBasedCheckpointFileManager takes the following to be created:
-
Checkpoint directory (Hadoop’s Path)
-
Configuration (Hadoop’s Configuration)
FileSystemBasedCheckpointFileManager initializes the internal properties.
Internal Properties
| Name | Description |
|---|---|
|
Hadoop’s FileSystem of the checkpoint directory |