Package crawlercommons.urlfrontier

Interface Urlfrontier.URLInfoOrBuilder

All Superinterfaces:
com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder
All Known Implementing Classes:
Urlfrontier.URLInfo, Urlfrontier.URLInfo.Builder
Enclosing class:
Urlfrontier
public static interface Urlfrontier.URLInfoOrBuilder extends com.google.protobuf.MessageOrBuilder

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    containsMetadata(String key)
    * Arbitrary key / values stored alongside the URL.
    String
    getCrawlID()
    * crawl ID *
    com.google.protobuf.ByteString
    getCrawlIDBytes()
    * crawl ID *
    String
    getKey()
    * The key is used to put the URLs into queues, the value can be anything set by the client but would typically be the hostname, domain name or IP or the URL.
    com.google.protobuf.ByteString
    getKeyBytes()
    * The key is used to put the URLs into queues, the value can be anything set by the client but would typically be the hostname, domain name or IP or the URL.
    Map<String,Urlfrontier.StringList>
    getMetadata()
    Deprecated.
    int
    getMetadataCount()
    * Arbitrary key / values stored alongside the URL.
    Map<String,Urlfrontier.StringList>
    getMetadataMap()
    * Arbitrary key / values stored alongside the URL.
    Urlfrontier.StringList
    getMetadataOrDefault(String key, Urlfrontier.StringList defaultValue)
    * Arbitrary key / values stored alongside the URL.
    Urlfrontier.StringList
    getMetadataOrThrow(String key)
    * Arbitrary key / values stored alongside the URL.
    String
    getUrl()
    * URL *
    com.google.protobuf.ByteString
    getUrlBytes()
    * URL *

    Methods inherited from interface com.google.protobuf.MessageLiteOrBuilder

    isInitialized

    Methods inherited from interface com.google.protobuf.MessageOrBuilder

    findInitializationErrors, getAllFields, getDefaultInstanceForType, getDescriptorForType, getField, getInitializationErrorString, getOneofFieldDescriptor, getRepeatedField, getRepeatedFieldCount, getUnknownFields, hasField, hasOneof

  • Method Details

    • getUrl

      String getUrl()
       * URL *
      string url = 1;
      Returns:
      The url.

    • getUrlBytes

      com.google.protobuf.ByteString getUrlBytes()
       * URL *
      string url = 1;
      Returns:
      The bytes for url.

    • getKey

      String getKey()
       * The key is used to put the URLs into queues, the value can be anything set by the client but would typically be the hostname,
 domain name or IP or the URL. If not set, the service will use a sensible default like hostname.
      string key = 2;
      Returns:
      The key.

    • getKeyBytes

      com.google.protobuf.ByteString getKeyBytes()
       * The key is used to put the URLs into queues, the value can be anything set by the client but would typically be the hostname,
 domain name or IP or the URL. If not set, the service will use a sensible default like hostname.
      string key = 2;
      Returns:
      The bytes for key.

    • getMetadataCount

      int getMetadataCount()
       *
 Arbitrary key / values stored alongside the URL. Can be anything needed by the crawler like http status, source URL etc...
      map<string, .urlfrontier.StringList> metadata = 3;

    • containsMetadata

      boolean containsMetadata(String key)
       *
 Arbitrary key / values stored alongside the URL. Can be anything needed by the crawler like http status, source URL etc...
      map<string, .urlfrontier.StringList> metadata = 3;

    • getMetadata

      @Deprecated Map<String,Urlfrontier.StringList> getMetadata()
      Deprecated.
      Use getMetadataMap() instead.

    • getMetadataMap

      Map<String,Urlfrontier.StringList> getMetadataMap()
       *
 Arbitrary key / values stored alongside the URL. Can be anything needed by the crawler like http status, source URL etc...
      map<string, .urlfrontier.StringList> metadata = 3;

    • getMetadataOrDefault

      Urlfrontier.StringList getMetadataOrDefault(String key, Urlfrontier.StringList defaultValue)
       *
 Arbitrary key / values stored alongside the URL. Can be anything needed by the crawler like http status, source URL etc...
      map<string, .urlfrontier.StringList> metadata = 3;

    • getMetadataOrThrow

      Urlfrontier.StringList getMetadataOrThrow(String key)
       *
 Arbitrary key / values stored alongside the URL. Can be anything needed by the crawler like http status, source URL etc...
      map<string, .urlfrontier.StringList> metadata = 3;

    • getCrawlID

      String getCrawlID()
       * crawl ID *
      string crawlID = 4;
      Returns:
      The crawlID.

    • getCrawlIDBytes

      com.google.protobuf.ByteString getCrawlIDBytes()
       * crawl ID *
      string crawlID = 4;
      Returns:
      The bytes for crawlID.