public static final class URLFrontierGrpc.URLFrontierBlockingStub extends io.grpc.stub.AbstractBlockingStub<URLFrontierGrpc.URLFrontierBlockingStub>
| Modifier and Type | Method and Description |
|---|---|
Urlfrontier.Empty |
blockQueueUntil(Urlfrontier.BlockQueueParams request)
Block a queue from sending URLs; the argument is the number of seconds of UTC time since Unix epoch
1970-01-01T00:00:00Z.
|
protected URLFrontierGrpc.URLFrontierBlockingStub |
build(io.grpc.Channel channel,
io.grpc.CallOptions callOptions) |
Urlfrontier.Integer |
deleteQueue(Urlfrontier.String request)
Delete the queue based on the key in parameter, returns the number of URLs removed this way *
|
Urlfrontier.Stats |
getStats(Urlfrontier.String request)
Return stats for a specific queue or the whole crawl if the value if empty or null *
|
java.util.Iterator<Urlfrontier.URLInfo> |
getURLs(Urlfrontier.GetParams request)
Stream URLs due for fetching from M queues with up to N items per queue *
|
Urlfrontier.StringList |
listQueues(Urlfrontier.Integer request)
Return the names of up to N active queues
a queue is active if it has URLs due for fetching; the default value of 0 sets no limit to the number of queue names to return *
|
Urlfrontier.Empty |
setActive(Urlfrontier.Boolean request)
De/activate the crawl.
|
Urlfrontier.Empty |
setDelay(Urlfrontier.QueueDelayParams request)
Set a delay from a given queue.
|
protected URLFrontierGrpc.URLFrontierBlockingStub build(io.grpc.Channel channel, io.grpc.CallOptions callOptions)
build in class io.grpc.stub.AbstractStub<URLFrontierGrpc.URLFrontierBlockingStub>public Urlfrontier.StringList listQueues(Urlfrontier.Integer request)
Return the names of up to N active queues a queue is active if it has URLs due for fetching; the default value of 0 sets no limit to the number of queue names to return *
public java.util.Iterator<Urlfrontier.URLInfo> getURLs(Urlfrontier.GetParams request)
Stream URLs due for fetching from M queues with up to N items per queue *
public Urlfrontier.Stats getStats(Urlfrontier.String request)
Return stats for a specific queue or the whole crawl if the value if empty or null *
public Urlfrontier.Integer deleteQueue(Urlfrontier.String request)
Delete the queue based on the key in parameter, returns the number of URLs removed this way *
public Urlfrontier.Empty blockQueueUntil(Urlfrontier.BlockQueueParams request)
Block a queue from sending URLs; the argument is the number of seconds of UTC time since Unix epoch 1970-01-01T00:00:00Z. The default value of 0 will unblock the queue. The block will get removed once the time indicated in argument is reached. This is useful for cases where a server returns a Retry-After for instance.
public Urlfrontier.Empty setActive(Urlfrontier.Boolean request)
De/activate the crawl. GetURLs will not return anything until SetActive is set to true. PutURLs will still take incoming data. *
public Urlfrontier.Empty setDelay(Urlfrontier.QueueDelayParams request)
Set a delay from a given queue. No URLs will be obtained via GetURLs for this queue until the number of seconds specified has elapsed since the last time URLs were retrieved. Usually informed by the delay setting of robots.txt.
Copyright © 2020-2021 Crawler-Commons. All Rights Reserved.