RSparkSubmit

yaml
type: "io.kestra.plugin.gcp.dataproc.batches.RSparkSubmit"

Examples

yaml
id: gcp_dataproc_r_spark_submit
namespace: company.team
tasks:
  - id: r_spark_submit
    type: io.kestra.plugin.gcp.dataproc.batches.RSparkSubmit
    mainRFileUri: 'gs://spark-jobs-kestra/dataframe.r'
    name: test-rspark
    region: europe-west3

Properties

mainRFileUri *string

name *string

region *string

archiveUris array

SubTypestring

args array

SubTypestring

execution

Definitions

io.kestra.plugin.gcp.dataproc.batches.AbstractBatch-ExecutionConfiguration

kmsKeystring

networkTagsarray

SubTypestring

networkUristring

serviceAccountEmailstring

subnetworkUristring

fileUris array

SubTypestring

impersonatedServiceAccount string

jarFileUris array

SubTypestring

peripherals

Definitions

io.kestra.plugin.gcp.dataproc.batches.AbstractBatch-PeripheralsConfiguration

metastoreServicestring

sparkHistoryServer

io.kestra.plugin.gcp.dataproc.batches.AbstractBatch-SparkHistoryServerConfiguration

dataprocClusterstring

projectId string

runtime

Definitions

io.kestra.plugin.gcp.dataproc.batches.AbstractBatch-RuntimeConfiguration

containerImagestring

propertiesobject

SubTypestring

versionstring

scopes array

SubTypestring

Default["https://www.googleapis.com/auth/cloud-platform"]

serviceAccount string

Outputs

state string

Possible Values

STATE_UNSPECIFIEDPENDINGRUNNINGCANCELLINGCANCELLEDSUCCEEDEDFAILEDUNRECOGNIZED

Core Plugins and tasks

Tasks that provide Kestra's built-in orchestration, I/O, and observability capabilities.

CoreStorageScript

PostgreSQL

Tasks that connect to PostgreSQL via JDBC to run queries and loads.

Database

SurrealDB

Tasks that run SurrealQL queries against SurrealDB.

Database

RSparkSubmit

More Plugins in this Category

Core Plugins and tasks

PostgreSQL

SurrealDB

2.0.0