com.databricks.sdk.service.pipelines

Class PipelinesAPI

java.lang.Object

com.databricks.sdk.service.pipelines.PipelinesAPI

@Generated
public class PipelinesAPI
extends Object

The Delta Live Tables API allows you to create, edit, delete, start, and view details about pipelines.

Delta Live Tables is a framework for building reliable, maintainable, and testable data processing pipelines. You define the transformations to perform on your data, and Delta Live Tables manages task orchestration, cluster management, monitoring, data quality, and error handling.

Instead of defining your data pipelines using a series of separate Apache Spark tasks, Delta Live Tables manages how your data is transformed based on a target schema you define for each processing step. You can also enforce data quality with Delta Live Tables expectations. Expectations allow you to define expected data quality and specify how to handle records that fail those expectations.

Constructor Summary

Constructors

Constructor and Description
PipelinesAPI(ApiClient apiClient) Regular-use constructor
PipelinesAPI(PipelinesService mock) Constructor for mocks

Method Summary

Modifier and Type	Method and Description
CreatePipelineResponse	create(CreatePipeline request) Create a pipeline.
void	delete(DeletePipelineRequest request) Delete a pipeline.
void	delete(String pipelineId)
GetPipelineResponse	get(GetPipelineRequest request) Get a pipeline.
GetPipelineResponse	get(String pipelineId)
GetPipelinePermissionLevelsResponse	getPipelinePermissionLevels(GetPipelinePermissionLevelsRequest request) Get pipeline permission levels.
GetPipelinePermissionLevelsResponse	getPipelinePermissionLevels(String pipelineId)
PipelinePermissions	getPipelinePermissions(GetPipelinePermissionsRequest request) Get pipeline permissions.
PipelinePermissions	getPipelinePermissions(String pipelineId)
GetUpdateResponse	getUpdate(GetUpdateRequest request) Get a pipeline update.
GetUpdateResponse	getUpdate(String pipelineId, String updateId)
PipelinesService	impl()
Iterable<PipelineEvent>	listPipelineEvents(ListPipelineEventsRequest request) List pipeline events.
Iterable<PipelineEvent>	listPipelineEvents(String pipelineId)
Iterable<PipelineStateInfo>	listPipelines(ListPipelinesRequest request) List pipelines.
ListUpdatesResponse	listUpdates(ListUpdatesRequest request) List pipeline updates.
ListUpdatesResponse	listUpdates(String pipelineId)
Wait<GetPipelineResponse,Void>	reset(ResetRequest request) Reset a pipeline.
Wait<GetPipelineResponse,Void>	reset(String pipelineId)
PipelinePermissions	setPipelinePermissions(PipelinePermissionsRequest request) Set pipeline permissions.
PipelinePermissions	setPipelinePermissions(String pipelineId)
StartUpdateResponse	startUpdate(StartUpdate request) Queue a pipeline update.
StartUpdateResponse	startUpdate(String pipelineId)
Wait<GetPipelineResponse,Void>	stop(StopRequest request) Stop a pipeline.
Wait<GetPipelineResponse,Void>	stop(String pipelineId)
void	update(EditPipeline request) Edit a pipeline.
void	update(String pipelineId)
PipelinePermissions	updatePipelinePermissions(PipelinePermissionsRequest request) Update pipeline permissions.
PipelinePermissions	updatePipelinePermissions(String pipelineId)
GetPipelineResponse	waitGetPipelineIdle(String pipelineId)
GetPipelineResponse	waitGetPipelineIdle(String pipelineId, Duration timeout, Consumer<GetPipelineResponse> callback)
GetPipelineResponse	waitGetPipelineRunning(String pipelineId)
GetPipelineResponse	waitGetPipelineRunning(String pipelineId, Duration timeout, Consumer<GetPipelineResponse> callback)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PipelinesAPI

public PipelinesAPI(ApiClient apiClient)

Regular-use constructor

PipelinesAPI

public PipelinesAPI(PipelinesService mock)

Constructor for mocks

Method Detail

waitGetPipelineIdle

public GetPipelineResponse waitGetPipelineIdle(String pipelineId)
                                        throws TimeoutException

Throws:

TimeoutException

waitGetPipelineIdle

public GetPipelineResponse waitGetPipelineIdle(String pipelineId,
                                               Duration timeout,
                                               Consumer<GetPipelineResponse> callback)
                                        throws TimeoutException

Throws:

TimeoutException

waitGetPipelineRunning

public GetPipelineResponse waitGetPipelineRunning(String pipelineId)
                                           throws TimeoutException

Throws:

TimeoutException

waitGetPipelineRunning

public GetPipelineResponse waitGetPipelineRunning(String pipelineId,
                                                  Duration timeout,
                                                  Consumer<GetPipelineResponse> callback)
                                           throws TimeoutException

Throws:

TimeoutException

create

public CreatePipelineResponse create(CreatePipeline request)

Create a pipeline.

Creates a new data processing pipeline based on the requested configuration. If successful, this method returns the ID of the new pipeline.

delete

public void delete(String pipelineId)

delete

public void delete(DeletePipelineRequest request)

Delete a pipeline.

Deletes a pipeline.

get

public GetPipelineResponse get(String pipelineId)

get

public GetPipelineResponse get(GetPipelineRequest request)

Get a pipeline.

getPipelinePermissionLevels

public GetPipelinePermissionLevelsResponse getPipelinePermissionLevels(String pipelineId)

getPipelinePermissionLevels

public GetPipelinePermissionLevelsResponse getPipelinePermissionLevels(GetPipelinePermissionLevelsRequest request)

Get pipeline permission levels.

Gets the permission levels that a user can have on an object.

getPipelinePermissions

public PipelinePermissions getPipelinePermissions(String pipelineId)

getPipelinePermissions

public PipelinePermissions getPipelinePermissions(GetPipelinePermissionsRequest request)

Get pipeline permissions.

Gets the permissions of a pipeline. Pipelines can inherit permissions from their root object.

getUpdate

public GetUpdateResponse getUpdate(String pipelineId,
                                   String updateId)

getUpdate

public GetUpdateResponse getUpdate(GetUpdateRequest request)

Get a pipeline update.

Gets an update from an active pipeline.

listPipelineEvents

public Iterable<PipelineEvent> listPipelineEvents(String pipelineId)

listPipelineEvents

public Iterable<PipelineEvent> listPipelineEvents(ListPipelineEventsRequest request)

List pipeline events.

Retrieves events for a pipeline.

listPipelines

public Iterable<PipelineStateInfo> listPipelines(ListPipelinesRequest request)

List pipelines.

Lists pipelines defined in the Delta Live Tables system.

listUpdates

public ListUpdatesResponse listUpdates(String pipelineId)

listUpdates

public ListUpdatesResponse listUpdates(ListUpdatesRequest request)

List pipeline updates.

List updates for an active pipeline.

reset

public Wait<GetPipelineResponse,Void> reset(String pipelineId)

reset

public Wait<GetPipelineResponse,Void> reset(ResetRequest request)

Reset a pipeline.

Resets a pipeline.

setPipelinePermissions

public PipelinePermissions setPipelinePermissions(String pipelineId)

setPipelinePermissions

public PipelinePermissions setPipelinePermissions(PipelinePermissionsRequest request)

Set pipeline permissions.

Sets permissions on a pipeline. Pipelines can inherit permissions from their root object.

startUpdate

public StartUpdateResponse startUpdate(String pipelineId)

startUpdate

public StartUpdateResponse startUpdate(StartUpdate request)

Queue a pipeline update.

Starts or queues a pipeline update.

stop

public Wait<GetPipelineResponse,Void> stop(String pipelineId)

stop

public Wait<GetPipelineResponse,Void> stop(StopRequest request)

Stop a pipeline.

Stops a pipeline.

update

public void update(String pipelineId)

update

public void update(EditPipeline request)

Edit a pipeline.

Updates a pipeline with the supplied configuration.

updatePipelinePermissions

public PipelinePermissions updatePipelinePermissions(String pipelineId)

updatePipelinePermissions

public PipelinePermissions updatePipelinePermissions(PipelinePermissionsRequest request)

Update pipeline permissions.

Updates the permissions on a pipeline. Pipelines can inherit permissions from their root object.

impl

public PipelinesService impl()