Package nom.tam.util

Class BufferedDataInputStream

java.lang.Object

java.io.InputStream

java.io.FilterInputStream

java.io.BufferedInputStream

nom.tam.util.BufferedDataInputStream

All Implemented Interfaces:

java.io.Closeable, java.io.DataInput, java.lang.AutoCloseable, ArrayDataInput, FitsIO

public class BufferedDataInputStream
extends java.io.BufferedInputStream
implements ArrayDataInput

This class is intended for high performance I/O in scientific applications. It combines the functionality of the BufferedInputStream and the DataInputStream as well as more efficient handling of arrays. This minimizes the number of method calls that are required to read data. Informal tests of this method show that it can be as much as 10 times faster than using a DataInputStream layered on a BufferedInputStream for writing large arrays. The performance gain on scalars or small arrays will be less but there should probably never be substantial degradation of performance.

Many new read calls are added to allow efficient reading off array data. The read(Object o) call provides for reading a primitive array of arbitrary type or dimensionality. There are also reads for each type of one dimensional array.

Note that there is substantial duplication of code to minimize method invocations. E.g., the floating point read routines read the data as integer values and then convert to float. However the integer code is duplicated rather than invoked. There has been considerable effort expended to ensure that these routines are efficient, but they could easily be superceded if an efficient underlying I/O package were ever delivered as part of the basic Java libraries. [This has subsequently happened with the NIO package and in an ideal universe these classes would be rewritten to take advantage of NIO.]

Testing and timing routines are provided in the nom.tam.util.test.BufferedFileTester class. Version 1.1: October 12, 2000: Fixed handling of EOF to return partially read arrays when EOF is detected. Version 1.2: July 20, 2009: Added handling of very large Object arrays. Additional work is required to handle very large arrays generally.

Field Summary

Fields

Modifier and Type	Field	Description
private BufferDecoder	bufferDecoder
private static java.util.logging.Logger	LOG
private BufferPointer	sharedBuffer
private static int	SKIP_BUFFER_SIZE	size of the skip buffer (if it exists) skipBuf.
private byte[]	skipBuf	Skip the requested number of bytes.

Fields inherited from class java.io.BufferedInputStream

buf, count, marklimit, markpos, pos

Fields inherited from class java.io.FilterInputStream

in

Fields inherited from interface nom.tam.util.FitsIO

BITS_OF_1_BYTE, BITS_OF_2_BYTES, BITS_OF_3_BYTES, BITS_OF_4_BYTES, BITS_OF_5_BYTES, BITS_OF_6_BYTES, BITS_OF_7_BYTES, BYTE_1_OF_LONG_MASK, BYTE_2_OF_LONG_MASK, BYTE_3_OF_LONG_MASK, BYTE_4_OF_LONG_MASK, BYTE_MASK, BYTES_IN_BOOLEAN, BYTES_IN_BYTE, BYTES_IN_CHAR, BYTES_IN_DOUBLE, BYTES_IN_FLOAT, BYTES_IN_INTEGER, BYTES_IN_LONG, BYTES_IN_SHORT, DEFAULT_BUFFER_SIZE, HIGH_INTEGER_MASK, INTEGER_MASK, SHORT_MASK, SHORT_OF_LONG_MASK

Constructor Summary

Constructors

Constructor	Description
BufferedDataInputStream(java.io.InputStream o)	Create a BufferedInputStream based on an input stream.
BufferedDataInputStream(java.io.InputStream o, int bufLength)	Create a BufferedInputStream based on a input stream with a specified buffer size.

Method Summary

Modifier and Type	Method	Description
private int	eofCheck(java.io.EOFException e, int i, int start, int length)	For array reads return an EOF if unable to read any data.
private long	handleExceptionInSkip(long skip, java.io.IOException e)
int	read(boolean[] b)	Read an array of boolean's.
int	read(boolean[] b, int start, int length)	Read a segment of an array of boolean's.
int	read(byte[] obuf, int offset, int length)	Read a segment of an array of byte's.
int	read(char[] c)	Read an array of char's.
int	read(char[] c, int start, int length)	Read a segment of an array of char's.
int	read(double[] d)	Read an array of double's.
int	read(double[] d, int start, int length)	Read a segment of an array of double's.
int	read(float[] f)	Read an array of float's.
int	read(float[] f, int start, int length)	Read a segment of an array of float's.
int	read(int[] i)	Read an array of int's.
int	read(int[] i, int start, int length)	Read a segment of an array of int's.
int	read(long[] l)	Read a segment of an array of long's.
int	read(long[] l, int start, int length)	Read a segment of an array of long's.
int	read(short[] s)	Read an array of short's.
int	read(short[] s, int start, int length)	Read a segment of an array of short's.
int	readArray(java.lang.Object o)	Deprecated.
boolean	readBoolean()
byte	readByte()
private void	readBytesIntoSharedBuffer(int bytes)
char	readChar()
double	readDouble()
float	readFloat()
void	readFully(byte[] b)
void	readFully(byte[] b, int off, int len)	Read a buffer and signal an EOF if the requested elements cannot be read.
int	readInt()
long	readLArray(java.lang.Object o)	Read an object.
java.lang.String	readLine()	Deprecated. Use BufferedReader methods.
long	readLong()
int	readPrimitiveArray(java.lang.Object o)	Deprecated. use readLArray(Object) instead
short	readShort()
int	readUnsignedByte()
int	readUnsignedShort()
java.lang.String	readUTF()
void	skipAllBytes(int toSkip)	Skip the number of bytes.
void	skipAllBytes(long toSkip)	Skip the number of bytes.
int	skipBytes(int toSkip)
java.lang.String	toString()

Methods inherited from class java.io.BufferedInputStream

available, close, mark, markSupported, read, reset, skip

Methods inherited from class java.io.FilterInputStream

read

Methods inherited from class java.io.InputStream

nullInputStream, readAllBytes, readNBytes, readNBytes, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface nom.tam.util.ArrayDataInput

mark, read, reset, skip

Methods inherited from interface nom.tam.util.FitsIO

close

Field Detail

LOG

private static final java.util.logging.Logger LOG

SKIP_BUFFER_SIZE

private static final int SKIP_BUFFER_SIZE

size of the skip buffer (if it exists) skipBuf.

See Also:

Constant Field Values

sharedBuffer

private final BufferPointer sharedBuffer

bufferDecoder

private final BufferDecoder bufferDecoder

skipBuf

private byte[] skipBuf

Skip the requested number of bytes. This differs from the skip call in that it takes an long argument and will throw an end of file if the full number of bytes cannot be skipped.

Constructor Detail

BufferedDataInputStream

public BufferedDataInputStream(java.io.InputStream o)

Create a BufferedInputStream based on an input stream.

Parameters:

o - the input stream to use for reading.

BufferedDataInputStream

public BufferedDataInputStream(java.io.InputStream o,
                               int bufLength)

Create a BufferedInputStream based on a input stream with a specified buffer size.

Parameters:

o - the input stream to use for reading.

bufLength - the buffer length to use.

Method Detail

eofCheck

private int eofCheck(java.io.EOFException e,
                     int i,
                     int start,
                     int length)
              throws java.io.EOFException

For array reads return an EOF if unable to read any data.

Parameters:

e - the eof exception that happened.

i - the current index

start - the start index

length - the element length

Returns:

the number of bytes read before the end of file exception.

Throws:

java.io.EOFException - if no extra bytes could be read

read

public int read(boolean[] b)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read an array of boolean's.

Specified by:

read in interface ArrayDataInput

Parameters:

b - array of boolean's.

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(boolean[] b,
                int start,
                int length)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a segment of an array of boolean's.

Specified by:

read in interface ArrayDataInput

Parameters:

b - array of boolean's.

start - start index in the array

length - number of array elements to read

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(byte[] obuf,
                int offset,
                int length)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a segment of an array of byte's.

Specified by:

read in interface ArrayDataInput

Overrides:

read in class java.io.BufferedInputStream

Parameters:

obuf - array of byte's.

offset - start index in the array

length - number of array elements to read

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

See Also:

ArrayDataInput.readFully(byte[], int, int)

read

public int read(char[] c)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read an array of char's.

Specified by:

read in interface ArrayDataInput

Parameters:

c - array of char's.

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(char[] c,
                int start,
                int length)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a segment of an array of char's.

Specified by:

read in interface ArrayDataInput

Parameters:

c - array of char's.

start - start index in the array

length - number of array elements to read

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(double[] d)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read an array of double's.

Specified by:

read in interface ArrayDataInput

Parameters:

d - array of double's.

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(double[] d,
                int start,
                int length)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a segment of an array of double's.

Specified by:

read in interface ArrayDataInput

Parameters:

d - array of double's.

start - start index in the array

length - number of array elements to read

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(float[] f)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read an array of float's.

Specified by:

read in interface ArrayDataInput

Parameters:

f - array of float's.

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(float[] f,
                int start,
                int length)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a segment of an array of float's.

Specified by:

read in interface ArrayDataInput

Parameters:

f - array of float's.

start - start index in the array

length - number of array elements to read

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(int[] i)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read an array of int's.

Specified by:

read in interface ArrayDataInput

Parameters:

i - array of int's.

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(int[] i,
                int start,
                int length)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a segment of an array of int's.

Specified by:

read in interface ArrayDataInput

Parameters:

i - array of int's.

start - start index in the array

length - number of array elements to read

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(long[] l)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a segment of an array of long's.

Specified by:

read in interface ArrayDataInput

Parameters:

l - array of long's.

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(long[] l,
                int start,
                int length)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a segment of an array of long's.

Specified by:

read in interface ArrayDataInput

Parameters:

l - array of long's.

start - start index in the array

length - number of array elements to read

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(short[] s)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read an array of short's.

Specified by:

read in interface ArrayDataInput

Parameters:

s - array of short's.

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

read

public int read(short[] s,
                int start,
                int length)
         throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a segment of an array of short's.

Specified by:

read in interface ArrayDataInput

Parameters:

s - array of short's.

start - start index in the array

length - number of array elements to read

Returns:

number of bytes read.

Throws:

java.io.IOException - if one of the underlying read operations failed

readArray

@Deprecated
public int readArray(java.lang.Object o)
              throws java.io.IOException

Deprecated.

Description copied from interface: ArrayDataInput

Read a generic (possibly multidimensional) primitive array. An Object[] array is also a legal argument if each element of the array is a legal.

The ArrayDataInput classes do not support String input since it is unclear how one would read in an Array of strings.

Specified by:

readArray in interface ArrayDataInput

Parameters:

o - A [multidimensional] primitive (or Object) array.

Returns:

number of bytes read.

Throws:

java.io.IOException - if the underlying stream failed

readBoolean

public boolean readBoolean()
                    throws java.io.IOException

Specified by:

readBoolean in interface java.io.DataInput

Throws:

java.io.IOException

readByte

public byte readByte()
              throws java.io.IOException

Specified by:

readByte in interface java.io.DataInput

Throws:

java.io.IOException

readBytesIntoSharedBuffer

private void readBytesIntoSharedBuffer(int bytes)
                                throws java.io.IOException,
                                       java.io.EOFException

Throws:

java.io.IOException

java.io.EOFException

readChar

public char readChar()
              throws java.io.IOException

Specified by:

readChar in interface java.io.DataInput

Throws:

java.io.IOException

readDouble

public double readDouble()
                  throws java.io.IOException

Specified by:

readDouble in interface java.io.DataInput

Throws:

java.io.IOException

readFloat

public float readFloat()
                throws java.io.IOException

Specified by:

readFloat in interface java.io.DataInput

Throws:

java.io.IOException

readFully

public void readFully(byte[] b)
               throws java.io.IOException

Specified by:

readFully in interface java.io.DataInput

Throws:

java.io.IOException

readFully

public void readFully(byte[] b,
                      int off,
                      int len)
               throws java.io.IOException

Description copied from interface: ArrayDataInput

Read a buffer and signal an EOF if the requested elements cannot be read. This differs from read(b,off,len) since that call will not signal and end of file unless no bytes can be read. However both of these routines will attempt to fill their buffers completely.

Specified by:

readFully in interface ArrayDataInput

Specified by:

readFully in interface java.io.DataInput

Parameters:

b - The input buffer.

off - The requested offset into the buffer.

len - The number of bytes requested.

Throws:

java.io.IOException

readInt

public int readInt()
            throws java.io.IOException

Specified by:

readInt in interface java.io.DataInput

Throws:

java.io.IOException

readLArray

public long readLArray(java.lang.Object o)
                throws java.io.IOException

Description copied from interface: ArrayDataInput

Read an object. An EOF will be signaled if the object cannot be fully read. This version works even if the underlying data is more than 2 Gigabytes.

Specified by:

readLArray in interface ArrayDataInput

Parameters:

o - The object to be read. This object should be a primitive (possibly multi-dimensional) array.

Returns:

number of bytes read.

Throws:

java.io.IOException - if the underlying stream failed

readLine

@Deprecated
public java.lang.String readLine()
                          throws java.io.IOException

Deprecated.

Use BufferedReader methods.

Emulate the deprecated DataInputStream.readLine() method. Originally we used the method itself, but Alan Brighton suggested using a java.io.BufferedReader to eliminate the deprecation warning. This was used for a long time, but more recently we noted that this doesn't work. We now use a simple method that largely ignores character encodings and only uses the "\n" as the line separator. This method is slow regardless. In the current version

Specified by:

readLine in interface java.io.DataInput

Returns:

The String read.

Throws:

java.io.IOException

readLong

public long readLong()
              throws java.io.IOException

Specified by:

readLong in interface java.io.DataInput

Throws:

java.io.IOException

readPrimitiveArray

@Deprecated
public int readPrimitiveArray(java.lang.Object o)
                       throws java.io.IOException

Deprecated.

use readLArray(Object) instead

This routine provides efficient reading of arrays of any primitive type. It is an error to invoke this method with an object that is not an array of some primitive type. Note that there is no corresponding capability to writePrimitiveArray in BufferedDataOutputStream to read in an array of Strings.

Parameters:

o - The object to be read. It must be an array of a primitive type, or an array of Object's.

Returns:

number of bytes read.

Throws:

java.io.IOException - if the underlying read operation fails

readShort

public short readShort()
                throws java.io.IOException

Specified by:

readShort in interface java.io.DataInput

Throws:

java.io.IOException

readUnsignedByte

public int readUnsignedByte()
                     throws java.io.IOException

Specified by:

readUnsignedByte in interface java.io.DataInput

Throws:

java.io.IOException

readUnsignedShort

public int readUnsignedShort()
                      throws java.io.IOException

Specified by:

readUnsignedShort in interface java.io.DataInput

Throws:

java.io.IOException

readUTF

public java.lang.String readUTF()
                         throws java.io.IOException

Specified by:

readUTF in interface java.io.DataInput

Throws:

java.io.IOException

skipAllBytes

public void skipAllBytes(int toSkip)
                  throws java.io.IOException

Description copied from interface: ArrayDataInput

Skip the number of bytes. This differs from the skip method in that it will throw an EOF if a forward skip cannot be fully accomplished... (However that isn't supposed to happen with a random access file, so there is probably no operational difference).

Specified by:

skipAllBytes in interface ArrayDataInput

Parameters:

toSkip - the number of bytes to skip

Throws:

java.io.IOException - if the underlying stream failed

skipAllBytes

public void skipAllBytes(long toSkip)
                  throws java.io.IOException

Description copied from interface: ArrayDataInput

Skip the number of bytes. This differs from the skip method in that it will throw an EOF if a forward skip cannot be fully accomplished... (However that isn't supposed to happen with a random access file, so there is probably no operational difference).

Specified by:

skipAllBytes in interface ArrayDataInput

Parameters:

toSkip - the number of bytes to skip

Throws:

java.io.IOException - if the underlying stream failed

handleExceptionInSkip

private long handleExceptionInSkip(long skip,
                                   java.io.IOException e)
                            throws java.io.IOException

Throws:

java.io.IOException

skipBytes

public int skipBytes(int toSkip)
              throws java.io.IOException

Specified by:

skipBytes in interface java.io.DataInput

Throws:

java.io.IOException

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object