BroadcastFactory — Pluggable Broadcast Variable Factories

BroadcastFactory is the contract for factories of broadcast variables in Apache Spark.

Note
As of Spark 2.0, it is no longer possible to plug a custom BroadcastFactory in, and TorrentBroadcastFactory is the one and only known implementation.

BroadcastFactory is exclusively used and instantiated inside of BroadcastManager.

package org.apache.spark.broadcast

trait BroadcastFactory {
  def initialize(isDriver: Boolean, conf: SparkConf, securityMgr: SecurityManager): Unit
  def newBroadcast[T: ClassTag](value: T, isLocal: Boolean, id: Long): Broadcast[T]
  def unbroadcast(id: Long, removeFromDriver: Boolean, blocking: Boolean): Unit
  def stop(): Unit
}
Note
BroadcastFactory is a private[spark] contract.
Table 1. BroadcastFactory Contract
Method Description

initialize

Used when…​FIXME

newBroadcast

Used when…​FIXME

unbroadcast

Used when…​FIXME

stop

Used when…​FIXME

Note
TorrentBroadcastFactory is the one and only known implementation of BroadcastFactory contract in Apache Spark.

results matching ""

    No results matching ""