Network Working Group A. Arcieri, Ed.
Internet-Draft PDTP.org
Expires: January 4, 2005 July 6, 2004
Peer Distributed Transfer Protocol
draft-arcieri-peer-distributed-transfer-protocol
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at http://
www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on January 4, 2005.
Copyright Notice
Copyright (C) The Internet Society (2004). All Rights Reserved.
Abstract
The Peer Distributed Transfer Protocol (PDTP) provides a method of
transferring files using peers to aid in distribution of content,
similar to . PDTP servers
export a dynamically changing directory heirarchy, making it somewhat
more like HTTP or FTP, and support a number of other features such as
metadata rich directory listings and support for file integrity
validation through the use of DSS signatures, and large networks
designed to replace FTP mirroring.
Arcieri Expires January 4, 2005 [Page 1]
Internet-Draft PDTP Protocol Specification July 2004
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Integer abbreviations . . . . . . . . . . . . . . . . . . 3
1.3 Handshaking . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Transaction format . . . . . . . . . . . . . . . . . . . . 5
2. Client/Server Protocol . . . . . . . . . . . . . . . . . . 7
2.1 Control transactions . . . . . . . . . . . . . . . . . . . 7
2.1.1 Command 0x1: Download File . . . . . . . . . . . . . . . . 7
2.1.2 Command 0x2: End Download . . . . . . . . . . . . . . . . 8
2.1.3 Command 0x3: Retrieve Server Certificate . . . . . . . . . 8
2.1.4 Command 0x4: Change Listener Port . . . . . . . . . . . . 8
2.1.5 Command 0x5: Set Maximum Download Concurrency . . . . . . 9
2.1.6 Command 0x6: Set Maximum Upload Concurrency . . . . . . . 9
2.1.7 Command 0x100: Directory Status . . . . . . . . . . . . . 10
2.1.8 Command 0x101: File Status . . . . . . . . . . . . . . . . 10
2.1.9 Command 0x102: Directory Listing . . . . . . . . . . . . . 11
2.1.10 Command 0x103: File Information . . . . . . . . . . . . . 12
2.1.11 Command 0x104: RDL Directory Listing . . . . . . . . . . . 13
2.1.12 Command 0x105: RDL File Information . . . . . . . . . . . 14
2.2 Transfer management . . . . . . . . . . . . . . . . . . . 14
2.2.1 Transfer management transactions (originating from
server) . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.2 Transfer management transactions (originating from
client) . . . . . . . . . . . . . . . . . . . . . . . . . 16
3. Piece Transfer Protocol . . . . . . . . . . . . . . . . . 18
4. Hub Protocol . . . . . . . . . . . . . . . . . . . . . . . 21
4.1 Handshaking . . . . . . . . . . . . . . . . . . . . . . . 21
4.2 Transactions originating from hub . . . . . . . . . . . . 22
4.2.1 Command 0x4: Change Listener Port . . . . . . . . . . . . 22
4.2.2 Command 0x13: Authorize Piece Download Attempt . . . . . . 22
4.2.3 Command 0x200: Add Handle . . . . . . . . . . . . . . . . 22
4.2.4 Command 0x201: Remove Handle . . . . . . . . . . . . . . . 23
4.3 Transactions originating from server . . . . . . . . . . . 23
5. Proxy Protocol . . . . . . . . . . . . . . . . . . . . . . 24
6. Acknowledgments . . . . . . . . . . . . . . . . . . . . . 25
References . . . . . . . . . . . . . . . . . . . . . . . . 26
Author's Address . . . . . . . . . . . . . . . . . . . . . 26
A. Path format . . . . . . . . . . . . . . . . . . . . . . . 27
B. Authorization mechanisms . . . . . . . . . . . . . . . . . 28
Intellectual Property and Copyright Statements . . . . . . 29
Arcieri Expires January 4, 2005 [Page 2]
Internet-Draft PDTP Protocol Specification July 2004
1. Introduction
1.1 Overview
This document describes version 1 of PDTP. Version 1 currently only
allows for operation on IPv4 networks. IPv6 support has been planned
from the start, and may or may not make it into version 1 depending
on the additional design complexity of facilitating transfers between
IPv4 and IPv6 hosts.
PDTP is actually a suite of four different protocol formats used by
different components of the system:
o The client/server protocol (Section 2), which is used by clients
to query the contents of a file repository and manages the file
transfer process. No file data is transferred using this protocol
directly. The PDTP client/server protocol's standard port number
is XXXX (pending IANA assignment).
o The hub protocol (Section 4), used by PDTP servers to talk to PDTP
hubs in order to query file repository contents, network mappings,
and various transfer regulation functions. The PDTP hub
protocol's standard port number is XXXX (pending IANA assignment).
o The piece transfer protocol (Section 3), or "peer protocol", used
by clients to transfer pieces to/from each other or from PDTP
hubs. The PDTP piece transfer protocol's standard port number is
XXXX (pending IANA assignment).
o The proxy protocol (Section 5), used by clients behind firewalls
to access PDTP servers. The PDTP proxy protocol's standard port
number is XXXX (pending IANA assignment).
1.2 Integer abbreviations
The following abbreviations will be used to describe various
integers:
uint8 Big-endian 8-bit unsigned integer
uint16 Big-endian 16-bit unsigned integer
uint32 Big-endian 32-bit unsigned integer
uint64 Big-endian 64-bit unsigned integer
Arcieri Expires January 4, 2005 [Page 3]
Internet-Draft PDTP Protocol Specification July 2004
1.3 Handshaking
Handshaking occurs when a PDTP client connects to a PDTP server:
Client sends:
uint32 Magic number (0x50445450, "PDTP")
Server responds:
uint32 Magic number (0x50445450, "PDTP")
uint16 Number of available protocol versions
uint16 Minimum protocol version
uint16 Next available protocol version [Optional]
...
uint16 Maximum available protocol version [Optional]
- OR -
Server rebonds:
uint16 Magic number (0x5244526E, "REDR");
uint16 String length
string Name of destination host
uint16 Host port
- OR -
Server responds:
uint16 Magic number (0x41555448, "AUTH")
... Authorization process (See Appendix C)
uint32 Magic number (0x50445450, "PDTP")
uint16 Minimum protocol version
uint16 Maximum protocol version
The 'PDTP' response indicates the server has accepted the connection.
An 'AUTH' response may be issued prior to a 'PDTP' response if the
server requires authorization to login.
If the server wishes to redirect the client to a different named
server, it can send a REDR response, followed by the hostname, IPv4
address in dotted quad format, or IPv6 address of the remote server,
and the port number to redirect to.
The server and client arbitrate which protocol version to use by
having the client pick a version number from an ordered list. The
list consists of a 16-bit value indicating how many protocol versions
are supported, and an ordered list of 16-bit integers comprising the
specific protocol revisions supported.
Currently the only available protocol version is 1. Therefore this
Arcieri Expires January 4, 2005 [Page 4]
Internet-Draft PDTP Protocol Specification July 2004
list should consist only of two 16-bit integers, each set to 1, the
first to indicate that only one protocol version is available and the
second to indicate that version 1 is the only supported version.
Client responds:
uint16 Protocol version
The client then selects which protocol version it would like to use.
If the client selects a protocol version which wasn't in the list,
the client has violated the protocol and the server should terminate
the connection.
1.4 Transaction format
All communication following the handshake takes the form of
transactions. Transactions may originate from the server as well, in
which case the serial number of the response should be set to zero.
All outstanding transactions from a client should have a unique
serial number so the client can determine which transaction a
particular response is associated with.
Client or server sends: [Required]
uint32 Transaction length
uint32 Transaction serial number
uint16 Transaction ID
uint16 Object count
Each transaction begins with a length field, which is the length of
the entire remainder of the transaction in bytes (i.e. the 4 bytes
for the transaction length value itself are not included) Thus the
smallest possible value for the transaction length is 8 bytes: 4
bytes for the serial number, 2 for the transaction ID, and two for
the object count. The entire transaction would be 12 bytes, counting
4 extra bytes for the transaction length value.
Every transaction has a type, which is specified in the "Transaction
ID" field. This transaction ID actually consists of a 1-bit boolean
value which indicates whether the transaction is 0: a command or 1: a
reply. The remaining 15 bits in the field comprise a big endian
integer identifier for the transaction. Specific types of
transactions will be enumerated below. This value is set to zero to
indicate an error when responding to a client's transaction.
Otherwise, all server responses will have the same ID and serial
number as the original client's transaction.
Arcieri Expires January 4, 2005 [Page 5]
Internet-Draft PDTP Protocol Specification July 2004
Transactions may or may not have a number of parameters known as
"objects". The number of objects contained within a transaction are
stored in the "Object count" field of the transaction header. The
length of all the objects and the object headers is counted in the
"Transaction length" value. Objects take the following format:
uint32 Object length
uint16 Object ID
string Object payload of specified length
What follows is a description of all transactions and associated
objects used by the various protocols in the PDTP suite. All objects
are mandatory unless stated otherwise. Objects should be arranged in
order of their respective numerical object identifiers from least to
greatest. This document arranges its object lists in such a manner;
transactions should be constructed in the same order presented in
this document in order to be valid.
Arcieri Expires January 4, 2005 [Page 6]
Internet-Draft PDTP Protocol Specification July 2004
2. Client/Server Protocol
2.1 Control transactions
2.1.1 Command 0x1: Download File
[Objects:]
Object ID: 0x2 (Path) [See Appendix A for format]
Payload: string Path to file
Object ID: 0x6 (Piece List) [Optional]
Payload: string Piece list
This transaction requests a download from the server. The piece list
consists of a list of uint32 pairs, which are the start and end of
ranges of piece indices we still need. So, for example, if we are
beginning a new download of a file with 2000 pieces, piece list would
be 8 bytes long and contain two uint32 values, 0 and 1999. If we are
getting the same file but have pieces 1000-1500, we would send 16
bytes which contain 4 uint32 values, 0, 999, 1501, and 1999. If we
have everything but the first piece, we would send 8 bytes which
contain two uint32 values, 0 and 0.
The piece list object may be optionally omitted, which indicates that
the client does not possess any pieces of the given file.
If the file handle and piece list are valid, the server will respond
with:
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
File transfers are stateful and server regulated. The server will
asynchronously send transfer management transactions for the given
file handle until the client sends the 0x2: End Download transaction
or the server sends the 0x17: End Transfer transaction.
In the event of an error, the server will respond with the following:
Object ID: 0x500 (Error message)
Payload: string Error message
Arcieri Expires January 4, 2005 [Page 7]
Internet-Draft PDTP Protocol Specification July 2004
2.1.2 Command 0x2: End Download
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
This transaction is sent by clients to terminate a transfer. It
should not be confused with transaction 0x16 (Terminate Transfer)
which is used by the server to inform clients that a download has
been terminated. This transaction should be sent by clients when
they are satisfied with the set of pieces they have received.
Otherwise, even after a client has successfully transferred all the
pieces of a file, the server will continue to include this file in
the piece selection process for uploaders, and if the client attempts
to download another file they can upload pieces from a file they've
already received successfully in order to earn the right to transfer
more pieces from a different file.
On success the server will respond with a transaction with no
objects, or on error it should respond with the following:
Object ID: 0x500 (Error message)
Payload: string Error message
2.1.3 Command 0x3: Retrieve Server Certificate
This transaction takes no objects from the client. The server
responds with:
Object ID: 0x40 (X.509 Certificate)
Payload: uint32 Server certificate containing DSA key
Administrators of PDTP servers may elect to allow the authenticity of
certain files to be verified using a DSS cryptographic signature.
This transaction allows retrieval of an X.509 certificate containing
the DSA key used to sign files on the server.
More information on X.509 can be found in RFC2459 [4].
2.1.4 Command 0x4: Change Listener Port
[Objects:]
Object ID: 0x12 (Port Number) [Optional]
Payload: uint16 New listener port number
Arcieri Expires January 4, 2005 [Page 8]
Internet-Draft PDTP Protocol Specification July 2004
This transaction allows a client to dynamically reassign their
listener port. By default, the PDTP server will assume the client is
capable of receiving connections on port 4045 unless specified
otherwise by this transaction. If a 0x12 (Port Number) object is not
passed in this transaction, then the client is set to passive mode.
If the port number is valid, the server will respond with a
transaction without any objects. In the event of an error, the
server will respond with the following:
Object ID: 0x500 (Error Message)
Payload: string Error message
2.1.5 Command 0x5: Set Maximum Download Concurrency
[Objects:]
Object ID: 0x7 (Concurrency Value)
Payload: uint16 New maximum concurrent connections
By default clients will be assigned a server side default for the
maximum number of concurrent piece downloads they wish to perform.
Clients may specify this value explicitly using this transaction. A
value of zero is not allowed, and servers may elect to set a ceiling
for this value.
If the concurrency value is valid, the server will respond with a
transaction without any objects. In the event of an error, the
server will respond with the following:
Object ID: 0x500 (Error Message)
Payload: string Error message
2.1.6 Command 0x6: Set Maximum Upload Concurrency
[Objects:]
Object ID: 0x7 (Concurrency Value)
Payload: uint16 New maximum concurrent connections
By default clients will be assigned a server side default for the
maximum number of concurrent piece uploads they wish to perform.
Clients may specify this value explicity using this transaction. A
value of zero is not allowed, and servers may elect to set a ceiling
for this value.
Arcieri Expires January 4, 2005 [Page 9]
Internet-Draft PDTP Protocol Specification July 2004
If the concurrency value is valid, the server will respond with a
transaction without any objects. In the event of an error, the
server will respond with the following:
Object ID: 0x500 (Error Message)
Payload: string Error message
2.1.7 Command 0x100: Directory Status
[Objects:]
Object ID: 0x2 (Path) [See Appendix A for format]
Payload: string Path to directory
This transaction requests the status of the specified directory. The
server returns a transaction with the following object:
Object ID: 0x102 (Entry Timestamp)
Payload: string 48-bit timestamp value
The timestamp is a 48-bit string which can be broken down as follows:
uint16 Year
uint32 Year seconds
The timestamp format consists of an 16-bit integer year and a 32-bit
value representing the number of seconds since the beginning of that
year.
In the event of an error, the server will return the following:
Object ID: 0x500 (Error Message)
Payload: string Error message
2.1.8 Command 0x101: File Status
[Objects:]
Object ID: 0x2 (Path) [See Appendix A for format]
Payload: string Path to file
This transaction requests the status of the specified file. The
server returns a transaction with the following object:
Object ID: 0x102 (Entry Timestamp)
Payload: string 48-bit timestamp value
Arcieri Expires January 4, 2005 [Page 10]
Internet-Draft PDTP Protocol Specification July 2004
The timestamp is a 48-bit string which can be broken down as follows:
uint16 Year
uint32 Year seconds
The timestamp format consists of an 16-bit integer year and a 32-bit
value representing the number of seconds since the beginning of that
year.
In the event of an error, the server will return the following:
Object ID: 0x500 (Error Message)
Payload: string Error message
2.1.9 Command 0x102: Directory Listing
[Objects:]
Object ID: 0x2 (Path) [See Appendix A for format]
Payload: string Path to directory
The transaction requests a listing of the specified directory. The
server returns an arbitrary number of the following objects:
Object ID: 0x100 (Subdirectory Entry)
Payload: string Subdirectory information
Object ID: 0x101 (File Entry)
Payload: string File information
The subdirectory information string (0x100) should consist of the
following:
uint16 Modification timestamp: year
uint32 Modification timestamp: year seconds
uint16 Length of directory name
string Directory name
The file information string (0x101) should consist of the following:
uint16 Modification timestamp: year
uint32 Modification timestamp: year seconds
uint64 File length in bytes
uint16 Length of filename
string Filename
The timestamp format consists of an 16-bit integer year and a 32-bit
Arcieri Expires January 4, 2005 [Page 11]
Internet-Draft PDTP Protocol Specification July 2004
value representing the number of seconds since the beginning of that
year.
In the event of an error, the server will return the following:
Object ID: 0x500 (Error Message)
Payload: string Error message
2.1.10 Command 0x103: File Information
[Objects:]
Object ID: 0x2 (Path) [See Appendix A for format]
Payload: string Path to file
This transaction requests the information required to complete the
download process. The server responds with the following objects:
Object ID: 0x3 (File Length)
Payload: uint64 File length
Object ID: 0x4 (Piece Length)
Payload: uint32 Piece length
And either of the following objects:
Object ID: 0x20 (MD5 Hashes)
Payload: string Buffer of MD5 hashes for all pieces in file
- OR -
Object ID: 0x21 (SHA1 Hashes)
Payload: string Buffer of SHA1 hashes for all pieces in file
If a cryptographic signature is available for a given file it should
be included in the transaction as well:
Object ID: 0x41 (DSS Signature) [Optional]
Payload: string DSS signature of file
Or in the event of an error, the server responds with an error
transaction containing only the following object:
Object ID: 0x500 (Error Message)
Payload: string Error message
Arcieri Expires January 4, 2005 [Page 12]
Internet-Draft PDTP Protocol Specification July 2004
The 0x20 or 0x21 object is a list of hashes for each piece in the
file. The number of bytes in the 0x20 or 0x21 object will be the
number of bytes in a single hash digest * ceiling(file length / piece
length). So, for 0x20 (MD5) with a piece length of 8 and a file
length of 17 the length of the 0x20 object will be 48 bytes, which is
the length of a single MD5 hash (16 bytes) * 3.
Currently MD5 and SHA1 are the only supported hash ciphers, but the
protocol is designed so objects 0x22, 0x23, etc. are reserved for
future hash ciphers.
After receiving a list of hashes, the client should then compute
hashes for each piece in any pre-existing copy of the file and
compare them to the server-provided list to determine what pieces of
the file remain to be transferred. This list should be passed as the
0x6 (Piece List) object of the 0x1 (Download File) transaction.
The optional 0x41 (DSS Signature) object is the SHA1 hash of the file
encrypted with the DSA key contained within the server's certificate,
which may be retrieved with the 0x3 (Retrieve Server Certificate)
transaction. This is used to ensure that files on the server have
not been tampered with prior to downloading.
For detailed information about the MD5 cipher, please see RFC1321
[1]. For detailed information about the SHA1 cipher, please see
RFC3174 [7]. For more information on the DSS, please see FIPS186 [8]
and RFC2802 [6].
2.1.11 Command 0x104: RDL Directory Listing
[Objects:]
Object ID: 0x2 (Path) [See Appendix A for format]
Payload: string Path to directory
This transaction requests a Rich Directory Listing. The server
responds:
Object ID: 0x104 (RDL response)
Payload: string XML document containing RDL response
Or in the event of an error, the server responds with the following:
Object ID: 0x500 (Error Message)
Payload: string Error message
RDL is an XML directory listing format which may be used secondarily
to the 0x102 (Directory Listing) transaction in order to obtain a
Arcieri Expires January 4, 2005 [Page 13]
Internet-Draft PDTP Protocol Specification July 2004
metadata rich directory listing with extended file attributes. The
format of these results is not specified in this document, but may be
found at:
2.1.12 Command 0x105: RDL File Information
[Objects:]
Object ID: 0x2 (Path) [See Appendix A for format]
Payload: string Path to file
This transaction requests RDL attributes for a file. The server
responds:
Object ID: 0x104 (RDL response)
Payload: string XML document containing RDL response
Or in the event of an error, the server responds with the following:
Object ID: 0x500 (Error Message)
Payload: string Error message
This is a counterpart to transaction 0x103 (File Information) only
also returning extended file attributes from the 0x104 (RDL Directory
Listing) transaction.
2.2 Transfer management
2.2.1 Transfer management transactions (originating from server)
2.2.1.1 Command 0x10: Send Piece to Peer
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
Object ID: 0x6 (Piece ID)
Payload: uint32 Piece number
Object ID: 0x10 (IPv4 Address)
Payload: uint32 Address of remote peer
Object ID: 0x12 (Port Number)
Payload: uint16 Port number of remote peer
Arcieri Expires January 4, 2005 [Page 14]
Internet-Draft PDTP Protocol Specification July 2004
This transaction instructs one peer to connect to another and send
the specified piece from the specified file.
2.2.1.2 Command 0x11: Receive Piece from Peer
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
Object ID: 0x6 (Piece ID)
Payload: uint32 Piece number
Object ID: 0x10 (IPv4 Address)
Payload: uint32 Address of remote peer
Object ID: 0x12 (Port Number)
Payload: uint16 Port number of remote peer
This transaction instructs one peer to connect to another and receive
the specified piece from the specified file.
2.2.1.3 Command 0x16: Terminate Piece Transfer
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
Object ID: 0x6 (Piece ID)
Payload: uint32 Piece number
Object ID: 0x10 (IPv4 Address)
Payload: uint32 Address of remote peer
This transaction instructs a client which is uploading a piece to one
of its peers to terminate its upload. No response from the client
doing the uploading is expected. Instead, the server should wait for
a 0x15 (Report failed piece transfer) transaction originating from
the client which is downloading the piece. This command should not
be sent to the client which is downloading a piece; this is a
protocol violation and should be ignored (or noted in a log as a
protocol violation)
Arcieri Expires January 4, 2005 [Page 15]
Internet-Draft PDTP Protocol Specification July 2004
2.2.1.4 Command 0x17: Terminate File Transfer
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
This transaction informs a client that the file associated with the
given handle is no longer being offered by the server, and that the
client should terminate all attempts to receive that particular file.
2.2.2 Transfer management transactions (originating from client)
2.2.2.1 Command 0x12: Authorize Piece Upload Attempt
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
Object ID: 0x6 (Piece ID)
Payload: uint32 Piece number
Object ID: 0x10 (IPv4 Address)
Payload: uint32 Address of remote peer
This transaction informs the server that a peer has connected and is
requesting to upload the specified piece (using an opcode of 0x10).
The listening peer then and asks for server authorization for the
transfer to continue. The server will always respond with a
transaction without any objects. If the connection attempt is
authorized the response transaction will have its transaction ID set
to 0x12, or if the connection is unauthorized the server will send an
error response, that is that the transaction ID will be set to 0.
2.2.2.2 Command 0x13: Authorize Piece Download Attempt
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
Object ID: 0x6 (Piece ID)
Payload: uint32 Piece number
Object ID: 0x10 (IPv4 Address)
Payload: uint32 Address of remote peer
Arcieri Expires January 4, 2005 [Page 16]
Internet-Draft PDTP Protocol Specification July 2004
This transaction is identical to the one above (0x12) except it is
used when the remote peer requests a piece download (using opcode
0x11)
2.2.2.3 Command 0x14: Report Successful Piece Transfer
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
Object ID: 0x6 (Piece ID)
Payload: uint32 Piece number
Object ID: 0x10 (IPv4 Address)
Payload: uint32 Address of remote peer
This transaction informs the server that a piece transfer has
completed successfully. It should be sent by both peers involved in
a transfer every time a piece transfer completes successfully. The
server does not send a response to this transaction; it is used only
for the server's internal bookkeeping.
2.2.2.4 Command 0x15: Report Failed Piece Transfer
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
Object ID: 0x6 (Piece ID)
Payload: uint32 Piece number
Object ID: 0x10 (IPv4 Address)
Payload: uint32 Address of remote peer
This transaction is functionally equivalent to the one above (0x14)
except it is used to report a failed piece transfer. The server
sends no response.
Arcieri Expires January 4, 2005 [Page 17]
Internet-Draft PDTP Protocol Specification July 2004
3. Piece Transfer Protocol
In PDTP, pieces are transferred between a peer listening on a TCP
port, which we will hereafter refer to as the 'listener', and a peer
connecting to this TCP port, which we will hereafter refer to as the
'connecting peer'.
The piece transfer protocol is very simple by design. The connecting
peer begins the exchange by sending the following:
Connecting peer sends:
uint32 Magic number (0x50545050, "PTPP")
Listener responds:
uint32 Magic number (0x50545050, "PTPP")
uint16 Minimum protocol version
uint16 Maximum protocol version
Both the minimum and maximum protocol version should currently be 1.
In the future these values will be the lowest and highest protocol
versions supported by the server.
In the future, the response from the connecting peer will vary
depending on the protocol versions supported by the listener and the
underlying transport protocol. However, at the current time the
connecting peer will always respond with the following:
uint16 Protocol version
uint16 Operation code
The connecting peer selects which protocol version it would like to
use. This value must be greater than or equal to the lowest version
supported and less than or equal to the highest protocol version
supported by the server. Any values which don't meet this criteria
are a protocol violation, and the connection should be terminated by
the server. At this point in time, this value should always be 1.
The operation code will be one of the following values (note that
these correspond directly to the transaction IDs from the server
originated transactions to begin a piece transfer):
0x10: Piece upload request (connecting peer => listener)
0x11: Piece download request (listener => connecting peer)
Arcieri Expires January 4, 2005 [Page 18]
Internet-Draft PDTP Protocol Specification July 2004
After the protocol version and operation code, the connecting peer
will send the following values which describe exactly what piece from
what server is to be transferred:
uint32 IPv4 address of coordinating server in network byte order
uint16 Port of coordinating server
uint32 File handle ID
uint32 Piece ID
The listener should then report the connecting peer's request to the
coordinating server using the proper transaction from the "Transfer
Management Transactions" section above: either 0x12 for a piece
upload request or 0x13 for a piece download request.
If the connecting peer requests a piece upload, then then listener
should first authorize the connection with the server. If the server
does not give authorization for this transfer the listener should
close the connection. Otherwise if authorization is received it
should send the following:
uint32 Piece offset
uint32 Bytes to transfer
This allows the listener, which is downloading the piece, to resume
the transfer of a piece which has already been partially received by
specifying an offset (in bytes) from the beginning of the piece from
which to begin the transfer. If this value is equal to or greater
than the piece size, or if the offset plus the bytes to transfer
exceeds the piece size, then the connecting peer should disconnect
and report a failed transfer due to a protocol violation. Otherwise
it should begin sending the piece from the specified offset. The
bytes to transfer field allows clients to grab the beginning portion
of a piece if they believe that portion to have been "tainted" by a
malicious client.
If the connecting peer is requesting a piece download, then in
addition to the header already enumerated above it should send:
uint32 Piece offset
uint32 Bytes to transfer
The listener should then request authorization from the server, and
if the server does not authorize the transfer the listener should
close the connection. Otherwise the listener should begin sending the
piece from the specified offset.
If the connection is lost before the piece transfer completes, this
should be reported to the server using the appropriate transaction
Arcieri Expires January 4, 2005 [Page 19]
Internet-Draft PDTP Protocol Specification July 2004
from the "Transfer Management Transactions" section, 0x15 (Report
failed transfer).
If the piece transfer succeeds, both the connecting peer and listener
should report so to the server using the appropriate transaction from
the "Transfer Management Transactions" section, 0x14 (Report
successful transfer).
After a successful piece transfer, both the connecting peer and
listener have the option of closing the connection. Otherwise, the
listener can wait for the following:
uint16 Operation code
and the two peers can then begin another piece transfer if so
desired.
Arcieri Expires January 4, 2005 [Page 20]
Internet-Draft PDTP Protocol Specification July 2004
4. Hub Protocol
Unlike the client/server protocol, communication between servers and
hubs allows fully synchronous, bidirectional communication.
Consequently, both the server and hub will use serial numbers to keep
track of outstanding transactions. Serial numbers used by the server
and hub are distinct, and the server and hub distinguish requests
from replies by transaction types alone.
4.1 Handshaking
PDTP hubs provide directory structure information to a PDTP server or
cluster of PDTP servers. In order to function, a PDTP server must be
connected to a hub. A PDTP server should connect to one and only one
PDTP hub at a single time. The connection is made via TCP.
After connecting the server initiates the handshaking process.
Server sends:
uint32 Magic number (0x50445350, "PDSP")
Hub responds:
uint32 Magic number (0x50445450, "PDSP")
uint16 Minimum protocol version
uint16 Maximum protocol version
- OR -
Hub responds:
uint16 Magic number (0x41555448, "AUTH")
... Authorization process (See Appendix C)
uint32 Magic number (0x50445450, "PDSP")
uint16 Minimum protocol version
uint16 Maximum protocol version
The 'PDSP' response indicates the hub has accepted the connection.
An 'AUTH' response may be issued prior to a 'PDSP' response if the
hub requires authorization to allow a server to connect.
Both the minimum and maximum protocol version should currently be 1.
In the future these values will be the lowest and highest protocol
versions supported by the hub.
The server should respond with the following:
uint16 Protocol version
The server selects which protocol version it would like to use. This
Arcieri Expires January 4, 2005 [Page 21]
Internet-Draft PDTP Protocol Specification July 2004
value must be greater than or equal to the lowest version supported
and less than or equal to the highest protocol version supported by
the server. Any values which don't meet this criteria are a protocol
violation, and the connection should be terminated by the hub. At
this point in time, this value should always be 1.
After handshaking, the hub and server communicate using the standard
transaction format. The following transactions are supported:
4.2 Transactions originating from hub
4.2.1 Command 0x4: Change Listener Port
This transaction is documented in section 2.1. It is used to inform
the server that the hub's piece server is listening on a port other
than the one to which the server connected.
4.2.2 Command 0x13: Authorize Piece Download Attempt
This transaction is documented in section 2.2.2. It is used when a
client connects to the hub's piece server and is requesting a
particular piece. The hub requests authorization from the server, and
the server can either grant or deny the connecting client's request
for a particular piece.
4.2.3 Command 0x200: Add Handle
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
Object ID: 0x2 (Path) [See Appendix A for format]
Payload: string Path to file
Object ID: 0x6 (Piece ID)
Payload: uint16 Number of pieces in file
This transaction informs the server that the given handle is now
valid and consists of the given number of pieces. This transaction
should be sent with a serial number of zero, and no response is
returned.
Arcieri Expires January 4, 2005 [Page 22]
Internet-Draft PDTP Protocol Specification July 2004
4.2.4 Command 0x201: Remove Handle
[Objects:]
Object ID: 0x1 (File Handle)
Payload: uint32 File handle ID
This transaction informs the server that the given handle is no
longer valid. The server should terminate all transfers associated
with the given handle. This transaction should be sent with a serial
number of zero, and no response is returned.
4.3 Transactions originating from server
PDTP hubs also implement all directory listing transactions
implemented by servers and used by clients for querying directories.
The expected behavior of a PDTP server is that requests for directory
or file information sent to the server are redirected to the hub, but
that the 0x100 and 0x101 status queries can be used to implement a
caching behavior within the server, to mitigate the amount of
bandwidth used by the hub for handling these queries. Hubs should
therefore be able to answer the following queries authoritatively for
the files and directory maps they are serving:
Command 0x100: Directory Status
Command 0x101: File Status
Command 0x102: Directory Listing
Command 0x103: File Information
Command 0x104: RDL Directory Listing
Command 0x105: RDL File Information
Arcieri Expires January 4, 2005 [Page 23]
Internet-Draft PDTP Protocol Specification July 2004
5. Proxy Protocol
PDTP supports making outgoing server and peer connections as well as
redirection of incoming peer connections through a proxy server.
This is accomplished partially through dynamic protocol translation
within the proxy server and partially through special protocol
constructions for making outgoing connections. The PDTP proxy server
also listens on port 4045.
Requesting a connection through a proxy server is accomplished as
follows:
Client sends:
uint32 Magic number (0x50786350, "PXCP")
Proxy server responds:
uint32 Magic number (0x50786350, "PXCP")
uint16 Minimum protocol version
uint16 Maximum protocol version
Both the minimum and maximum protocol version should currently be 1.
In the future these values will be the lowest and highest protocol
versions supported by the server.
Client responds:
uint16 Protocol version
The client then selects which protocol version it would like to use.
This value must be greater than or equal to the lowest version
supported and less than or equal to the highest protocol version
supported by the server. Any values which don't meet this criteria
are a protocol violation, and the connection should be terminated by
the server.
After selecting the protocol version it would like to use, the client
sends the address and port of the remote system to connect to:
uint32 Host IP address in network byte order
uint16 Host port
The client may then issue a PDTP or PTPP connection request. If the
connection to the remote system fails the proxy should simply
terminate the connection. Otherwise it should connect to the
requested system and repeat the PDTP or PTPP connection requests made
by the client.
Arcieri Expires January 4, 2005 [Page 24]
Internet-Draft PDTP Protocol Specification July 2004
6. Acknowledgments
Thanks to Marshall Rose for his conversion tools from the RFC-2629
[5] XML format to HTML and RFC.
Arcieri Expires January 4, 2005 [Page 25]
Internet-Draft PDTP Protocol Specification July 2004
References
[1] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, April
1992.
[2] Krawczyk, H., Bellare, M. and R. Canetti, "HMAC: Keyed-Hashing
for Message Authentication", RFC 2104, February 1997.
[3] Yergeau, F., "UTF-8, a transformation format of ISO 10646", RFC
2279, January 1998.
[4] Housley, R., Ford, W., Polk, T. and D. Solo, "Internet X.509
Public Key Infrastructure Certificate and CRL Profile", RFC
2459, January 1999.
[5] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629, June
1999.
[6] Davidson, K. and Y. Kawatsura, "Digital Signatures for the v1.0
Internet Open Trading Protocol (IOTP)", RFC 2802, April 2000.
[7] Eastlake, D. and P. Jones, "US Secure Hash Algorithm 1 (SHA1)",
RFC 3174, September 2001.
[8] National Institute of Standards and Technology, "Digital
Signature Standard", FIPS PUB 186-1, December 1998, .
Author's Address
Anthony Arcieri (editor)
PDTP.org
Arcieri Expires January 4, 2005 [Page 26]
Internet-Draft PDTP Protocol Specification July 2004
Appendix A. Path format
All paths used by PDTP are canonical Unix format absolute paths,
given in the UTF-8 character set (see RFC2279 [3]).
Entities in a path are either directories are files. Every directory
name should be postfixed with a '/' character, with a degenerate form
of "/" for the root directory. Therefore, all paths should begin
with a '/' character, and when the last entity in a path is a
directory, the entire path should also end with a '/' character.
(Note: The traditional symbolic directory names '.' for the current
directory and '..' for the current directory's parent are not
allowed)
Files always come at the end of a list of directories. This
directory list must contain at least one element, the root directory
'/'. So, in order to reference a file named 'foo' in the root
directory, a client would pass a path of '/foo'.
The maximum length of a path is 1024 bytes. Paths longer than this
limit are a protocol violation and an appropriate error should be
returned.
Arcieri Expires January 4, 2005 [Page 27]
Internet-Draft PDTP Protocol Specification July 2004
Appendix B. Authorization mechanisms
The authorization process by the server sending the authorization
magic number:
uint32 Magic number (0x41555448, "AUTH")
The client then sends a 16-bit integer indicating the authorization
mechanism index to utilize.
uint16 Authorization mechanism
The server responds:
uint8 ACK (0x1) or NACK (0x0)
A response of ACK indicates the use of this mechanism is allowed, and
NACK rejects it. If the server responds NACK, the client may either
issue another authorization mechanism to try, or disconnect.
If the server responds ACK, the specified authorization process is
performed. At the end of the process, the server responds ACK (0x1)
if the authorization succeeded or NACK (0x0) if it failed. If an
authorization attempt fails, the client may send another
authorization mechanism index to attempt, or disconnect if it has
exhausted the mechanisms it wishes to use.
* Authorization mechanism 0x0: HMAC-SHA1
This mechanism utilizes RFC2104 [2] HMAC hashing, using SHA1 (RFC3174
[7]) as the underlying hash cipher. The authentication proceeds as
follows:
Server sends:
string 160-bits (20 bytes) of random challenge data
The message to be encrypted is 20 bytes of random challenge data
which is sent to the client by the server. An HMAC-style hash should
be applied to this message with an arbitrary length key common to
both parties. The authentication proceeds as follows:
Client responds:
string 160-bits of SHA1 digest resulting from HMAC hashing
The server should then check the hash and send ACK or NACK
accordingly depending on if the digest matches or not.
Arcieri Expires January 4, 2005 [Page 28]
Internet-Draft PDTP Protocol Specification July 2004
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
intellectual property or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; neither does it represent that it
has made any effort to identify any such rights. Information on the
IETF's procedures with respect to rights in standards-track and
standards-related documentation can be found in BCP-11. Copies of
claims of rights made available for publication and any assurances of
licenses to be made available, or the result of an attempt made to
obtain a general license or permission for the use of such
proprietary rights by implementors or users of this specification can
be obtained from the IETF Secretariat.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights which may cover technology that may be required to practice
this standard. Please address the information to the IETF Executive
Director.
Full Copyright Statement
Copyright (C) The Internet Society (2004). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assignees.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
Arcieri Expires January 4, 2005 [Page 29]
Internet-Draft PDTP Protocol Specification July 2004
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
Arcieri Expires January 4, 2005 [Page 30]