Add a Sequential ID Column to a Spark DataFrame — sdf_with_sequential

Add a sequential ID column to a Spark DataFrame. The Spark zipWithIndex function is used to produce these. This differs from sdf_with_unique_id in that the IDs generated are independent of partitioning.

Usage

sdf_with_sequential_id(x, id = "id", from = 1L)

Arguments

x: A spark_connection, ml_pipeline, or a tbl_spark.
id: The name of the column to host the generated IDs.
from: The starting value of the id column