crawlercommons.urlfrontier

Class URLFrontierGrpc.URLFrontierFutureStub

java.lang.Object

io.grpc.stub.AbstractStub<S>

io.grpc.stub.AbstractFutureStub<URLFrontierGrpc.URLFrontierFutureStub>

crawlercommons.urlfrontier.URLFrontierGrpc.URLFrontierFutureStub

Enclosing class:

URLFrontierGrpc

public static final class URLFrontierGrpc.URLFrontierFutureStub
extends io.grpc.stub.AbstractFutureStub<URLFrontierGrpc.URLFrontierFutureStub>

Nested Class Summary

Nested classes/interfaces inherited from class io.grpc.stub.AbstractStub

io.grpc.stub.AbstractStub.StubFactory<T extends io.grpc.stub.AbstractStub<T>>

Method Summary

Modifier and Type	Method and Description
com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Empty>	blockQueueUntil(Urlfrontier.BlockQueueParams request) Block a queue from sending URLs; the argument is the number of seconds of UTC time since Unix epoch 1970-01-01T00:00:00Z.
protected URLFrontierGrpc.URLFrontierFutureStub	build(io.grpc.Channel channel, io.grpc.CallOptions callOptions)
com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Integer>	deleteQueue(Urlfrontier.String request) Delete the queue based on the key in parameter, returns the number of URLs removed this way *
com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Boolean>	getActive(Urlfrontier.Empty request) Returns true if the crawl is active, false if it has been deactivated with SetActive(Boolean) *
com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Stats>	getStats(Urlfrontier.String request) Return stats for a specific queue or the whole crawl if the value if empty or null *
com.google.common.util.concurrent.ListenableFuture<Urlfrontier.QueueList>	listQueues(Urlfrontier.Pagination request) Return the names of up to N active queues a queue is active if it has URLs due for fetching; by default the service will return up to 100 results from offset 0 *
com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Empty>	setActive(Urlfrontier.Boolean request) De/activate the crawl.
com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Empty>	setDelay(Urlfrontier.QueueDelayParams request) Set a delay from a given queue.

Methods inherited from class io.grpc.stub.AbstractFutureStub

newStub, newStub

Methods inherited from class io.grpc.stub.AbstractStub

getCallOptions, getChannel, withCallCredentials, withChannel, withCompression, withDeadline, withDeadlineAfter, withExecutor, withInterceptors, withMaxInboundMessageSize, withMaxOutboundMessageSize, withOption, withWaitForReady

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

build

protected URLFrontierGrpc.URLFrontierFutureStub build(io.grpc.Channel channel,
                                                      io.grpc.CallOptions callOptions)

Specified by:

build in class io.grpc.stub.AbstractStub<URLFrontierGrpc.URLFrontierFutureStub>

listQueues

public com.google.common.util.concurrent.ListenableFuture<Urlfrontier.QueueList> listQueues(Urlfrontier.Pagination request)

 Return the names of up to N active queues
a queue is active if it has URLs due for fetching; by default the service will return up to 100 results from offset 0 *
 

getStats

public com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Stats> getStats(Urlfrontier.String request)

 Return stats for a specific queue or the whole crawl if the value if empty or null *
 

deleteQueue

public com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Integer> deleteQueue(Urlfrontier.String request)

 Delete the queue based on the key in parameter, returns the number of URLs removed this way *
 

blockQueueUntil

public com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Empty> blockQueueUntil(Urlfrontier.BlockQueueParams request)

 Block a queue from sending URLs; the argument is the number of seconds of UTC time since Unix epoch
1970-01-01T00:00:00Z. The default value of 0 will unblock the queue. The block will get removed once the time
indicated in argument is reached. This is useful for cases where a server returns a Retry-After for instance. 
 

setActive

public com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Empty> setActive(Urlfrontier.Boolean request)

 De/activate the crawl. GetURLs will not return anything until SetActive is set to true. PutURLs will still take incoming data. *
 

getActive

public com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Boolean> getActive(Urlfrontier.Empty request)

 Returns true if the crawl is active, false if it has been deactivated with SetActive(Boolean) *
 

setDelay

public com.google.common.util.concurrent.ListenableFuture<Urlfrontier.Empty> setDelay(Urlfrontier.QueueDelayParams request)

 Set a delay from a given queue.
No URLs will be obtained via GetURLs for this queue until the number of seconds specified has 
elapsed since the last time URLs were retrieved.
Usually informed by the delay setting of robots.txt.