$Id: NEWS 20889 2009-05-12 22:07:17Z jimg $ News for version 3.9.3 This is small fix for the DAS parser code which is used by the hdf4 handler when it caches Attributes (reading attributes from the hdf4 files can be fairly time comsuming). News for version 3.9.2 Memory leak fixed when using DDS::transfer_attributes on a DAS that uses the ALias keyword. News for version 3.9.1 The CE parser now returns an error if an array of structures that contains arrays is imporperly constrained (the enclosing structure array _may_ be subsampled, but that subsampling must be the same for each of its fields. The fields can be independently subsampled. A problem with the client-side cache has been fixed. Variable names can now be quoted! That is, in a CE you can say: "my odd name"."% H2O"[10:13] and it will parse. Must clients and all web browsers will encode all of this using the web's URL escape syntax, whic his hard to read, but this means that any character can now appear in a variable name (double quotes can be escaped using a backslash and backslashes can be includes using '\\'). The quotes are optional, of course, so all CEs that work now will continue to work. News for version 3.9.0 libdap now supports DAP 3.2. The evolving DAP 3.x protocol is described at http://docs.opendap.org/index.php/DAP3/4. The most important change from DAP 3.1 to 3.2 is: DAP 3.2 introduces the notion of protocol negotiation, similar to HTTP's response type or encoding negotiation. The client MAY send an XDAP-Accept header to tell the server the highest version of the protocol that the client can understand. The server MUST then respond using only responses at or below that version number of the protocol. Note that an Error response (e.g., "The response you requested cannot be returned using the protocol version you understand") can be understood by any client. If a client does not send the XDAP-Accept header, then the server MUST assume a DAP 2.0/3.1 client. The only functional difference between DAP 2.0, 3.0 and 3.1 is form/content of the version information returned in the HTTP response header. By allowing the server to respond with a lower version the protocol can support old servers (since that's how they will respond) and enable newer clients to treat the lower version responses as errors (because the newer servers can be expected to discriminate between different server versions). New servers SHOULD always return a response that conforms to the version sent from the client in the XDAP-Accept header. In addition to returning the DAP protocol version using the XDAP header (which is a mechanism specific to HTTP), the protocol version will also be returned in the DDX Dataset element using the XML attribute dap-version. See DDX. With this version of libdap the DDX is slightly different for DAP 3.2 - it now includes DAP protocol version information and an xmlbase element. There was ambiguity regarding how an array of structures would be constrained. The web page of DAP 2 Errata has been updated with a clarification of this and the constraint expression parser has been updated to reflect that fix. See the DAP3/4 page and ticket 975 for the complete info. Short version: Suppose 's' is an array of structures and 'm' is an array that is a member of 's'. You can constraint 'm' like this: s2[0:4].m[2:7] Now the notion that a dataset identifier (e.g., a file name) is bound to the DDS has been dropped and that identifier is bound to a variable. This lets the library be used in contexts where a DDS holds variables from several different places (e.g., files). String attribute values were always quoted (quotes were added if not present in the data set) and while this was OK for the DAS response, it broke the DDX. Now quotes are added to DAS response but not the values themselves. See ticket 1163. Nested Sequences were failing when the constraint forced one or more rows of the inner Sequence to be empty. Fixed. A number of build issues have been fixed in this version, see the ChangeLog file. News for version 3.8.2 Significant improvements to the HTTP cache. The cache software could return erroneous responses in some rare cases when using multi-threaded code. Fixed. Also the entire caching system is now much more robust since the 'table' that contains information about individual entries is encapsulated in it's own class. The HTTP cache is still both thread-safe and _not_ multi-process safe. However, it should be possible to modify the HTTPCacheTable class to use a database system like MySQL or SQLite to make it MP-safe. The 'ancillary information' functions have been moved to their own class (from DODSFilter and the file cgi_util.cc). I fixed the libdap win32 installer so that it does not install stuff in the win32 system directories anymore. News for version 3.8.1 The syntax for PROXY_SERVER and NO_PROXY_FOR have been fixed so that they are easier to use. PROXY_SERVER now takes a value like 'http://user:pw@squid.proxy.edu:3128' and uses it correctly. In that value, all parts but the host name (squid.proxy.edu) are optional and the old syntax, as well as several comman variants, are accepted. For the NO_PROXY_FOR entry, the ',' is now optional. Only the HTTP protocol is supported or the proxy server. News for version 3.8.0, 29 February 2008 The libdap classes and code are now inside of the libdap namespace. In order to access any of the classes, for example, you will need to do one of the following. After including the libdap headers you can: 1. add a using statement for the entire libdap namespace: using namespace libdap ; 2. add a using statement for the classes that you will be using: using libdap::DAS ; 3. inside your code scope the use of libdap classes. libdap::DAS *das = code_to_get_das() ; HTTPCache updated to cache an entry, returning not only the FILE pointer but also the name of the file in the cache. This way, the cached item could be passed to a data handler, such as the netcdf_handler, and read. The pkg build for Mac OSX was updated to automate the creation of the ReadMe.txt file, add a README and NEWS file to the dmg and automatically create the dmg using DropDMG. News for version 3.7.10, 28 November 2007 Fixed XDRStreamMarshaller so that it no longer allocates a huge buffer. This fixes a bug where the BES fails to start on smaller machines. News for version 3.7.9, 21 November 2007 Bumped up the soname version numbers for libdaoclient and libdapserver due to changes in the HTTPResponse and AlarmHandler interfaces. The transfer_data() method defined for Sequence and Structure has been generalized and implemented for all classes. In the process, so issues with the way nested sequences were handled have been fixed. I renamed the method to intern_data() since it is now a part of libdap (and not just two classes). The method uses the read() methods defined for the type classes (Byte, ..., Grid) to read data into variables in a DDS as if they were read in using deserialize(). This is used by the ASCII response generator and might be used by other 'formatted output' generators. Generated files (like the grammar files) are now shipped with the source distributions. (From Patrice Dumas) Methods which write output using the C++ iostream system have been added back into the library and are going to replace the FILE* versions of those methods. We have also added an 'Un/Marshaller' set of classes so that we can release a version of Hyrax (slated to be 1.4) that will improve BES/OLFS communication efficiency. The functionality of the Passive* type classes was subsumed by their parent classes and they were removed from the library to cut down on 'hierarchy clutter.' Some other unneeded files were also removed. The maximum size of a Str object was changes to 65535 (DODS_USHORT_INT-1) to accommodate HDF5 strings. Checkouts from subversion now have no (?) generated files. We are keeping a copy of the grammars in subdirectory named 'grammarfiles.' HTTP response codes are now available in libdap HTTPResponse objects. From Darren Hardy. News for version 3.7.8, 26 June 2007 Updated the email address for support to support at opendap.org and changed the tech email list address to opendap-tech at opendap.org. This is part of the plan to shift support services to OPeNDAP and to use the tech email list as part of that plan. Updated to the latest gnulib software. Memory errors fixed: In the HTTP processing code for clients which was triggered when a client read from an older server (older servers had malformed HTTP headers). An error in the regular expression class (Regex) where the build generated flawed code on a 64-bit machine. OSX Build improvements. Patrice Dumas' changes to dods-datatypes.h - Now the header uses the C99 types unless the stdint.h header is not present. pkg-config support. From a patch by Patrice Dumas. General improvements to comments, strings and error messages throughout the libraries. News for version 3.7.7, 2 May 2007 Fixed a bug where the build was not installing the dapserver and dapclient libraries. Fixed a handful of platform-specific build issues. There are some minor performance improvements to the constraint evaluator. Repaired some problems with the server-side functions. News for version 3.7.6, 12 March 2007 Fixed a bug in the linear_scale() Constraint Expression function when that function was used with plain arrays. I fixed a build issue on linux for ml-structs caused by a bad/missing #include in GnuRegex.h. News for version 3.7.5, 7 Feb 2007 Many bug fixes for the Server-Side functions linear_scale(), grid(), geogrid() and geoarray(). Added dump method to the BaseType classes, DAS, DDS, DataDDS. To do this, created a DapObj base class for all of these to inherit from (directly or indirectly) and in the DapObj class is the operator<< method. This will aid in debugging. Created an indentation classes to help with dumping objects. Win32 port fixes. The Win32 build is closer to the Unix build. Our hope is to get the two to be almost identical so that we can all build on Win32 and thus get away from having the win32 releases lag behind the Unix releases. Fix in Vector for gcc 4.1 News for version 3.7.4, 02 Jan 2007 Build enhancements and bug fixes. See ChangeLog for specifics. News for version 3.7.3, 24 Nov 2006 Fixed unescattr() so that it works! Rob added improvements to the win32 build. Fixed a number of bugs including: Problems with the DDS::transfer_attributes() method which broke libnc-dap; Added a new configuration parameter to .dodsrc so that SSL validation can be suppressed; Fixed problems with the change from \n to \r\n line terminators for MIME headers which slipped through the cracks the first time; and add gzip and compress to the set of accepted compression types supported by the client side. Added Connect::request_ddx() which asks a server for the DDX response. Previously, it was possible to use getdap to print the DDX, but that object was actually built using the DAS and DDS from a server. Now the server is asked for the DDX. Servers should support this response to be compliant with DAP 3.1. In the ConstraintEvaluator class, the add_function() method now allows a server to override a function from libdap with a new definition of the same name. I added a new interface for the scalar types: value() and set_value() can be used to get and set values. These are much simple to use than the older val2buf() and buf2val() methods. This idea was copied from copied from the PassiveByte, ..., classes. New server-side functions for geographical constraints have been added. These functions provide a way to select parts of Grid or Array variables using Latitude and Longitude. Also added is a version() function which can be used by clients to figure out which version of functions is present. The version function has two forms, one which returns the information as plain text and one which returns the information in a small XML document. libdap 3.7.3 is a beta release of these functions. Note that these are _server-side_ functions so before they can be used, servers need to be built using this library and installed. The dap-config script has been fixed so that it behaves more like other such scripts. News for version 3.7.2 Fixed a persistent bug in the GNURegex code. (ticket 389) Fixed a problem in HTTPConnect where the change from newline to cr/nl line terminators was not accommodated (no ticket, but part of the issues behind ticket 552). Added Sequence::transfer_data(). This method uses the read() method to read data and follows the same logic used by serialize to determine which data is 'sent' but instead records the data in the d_values field. Thus a Sequence can transfer data to itself. The locally stored data apes the ability of the other classes to store a complete response and is the basis for the new ASCII response code (in dap-server/asciival) which Server4 uses. There's a fair amount of the geogrid() code now in place, although the function does not work, I have added a GeoConstraint class which is close to complete. The CEFunctionsTest unit tests fail. News for version 3.7.1 Fixed bug #480. Attributes from the HDF4 server were not being handled in a way that made sense to netCDF clients. Added a new method to AttrTable so that a vector of strings can be used to set a vector of attribute values in one call. No more need to build a loop every time you want to set values. The grid() CE function now conforms to the design submitted to URI as part of the REASoN award. Fixed a bug when % signs were not handled correctly by the code in escaping.cc. News for version 3.7.0 The big change: The first of the DAP3/4 features was added to the libdap library; the library now includes the DAP protocol version number in all responses. A savvy client can test for this and process accordingly. A second feature, representation of the data source metadata in XML, is supported in alpha form. Expect the 'DDX' response to change slightly in the next release. The DDX returned now contains an obsolete element named 'Blob' which we won't be using. Other than the Blob element, the current DDX probably will not change much except for the addition of new datatypes, something that will take place only after other changes are made to the protocol. Many of the handlers now support returning the DDX, so developers can begin to play with it's capabilities. What's really nice about the DDX: It's cool that the DDX now encode the metadata in XML, but the really nice feature is that it combines information from the DDS and DAS, making the association of attributes to variables much easier. Updated the reference guide (HTML pages in docs). Fixed a pernicious bug in GNURegex.cc Added the class ConstraintEvaluator. The DDS class now takes the constraint evaluator as a parameter, so it's possible to replace ours with your own. The single library libdap has been split into three libraries: libdap for the DAP functionality, libdapclient for the client-side classes and libdapserver for the server-side classes. Added INSTALL.AIX which contains information about building libdap on AIX using the _native compiler_. News for version 3.6.2 This version includes two bug fixes. 1. The library can now correctly reads binary data objects saved using the web interface. Before, this was very hard to do. The utility getdap has been modified so that it can decode these saved '.dods' files. 2. The DODSFilter class used to build all of our data handlers no longer blocks when returning a data object to a web browser. This fixes a problem where the 'Get Binary' button on the data server's web interface would hang seemingly forever. In fact it did return the data blob, but only after the blocked handler had timed out. News for version 3.6.1 Fixed a bug in deflate.c: Some comments used the C++ style comments and gcc rejected that. Data server filters built with the 3.6.0 library were not compatible with the dap-server 3.6.0 software because the DODSFilter class did not recognize the -t option (which is used to pass the handler a timeout value). News for version 3.6.0 Added patches for RPM spec files from Patrice Dumas. Fixed a problem where Grids with two or more dimensions with the same name were flagged as broken (by Grid::check_semantics). Now they are allowed as per the specification. Added a new method get_parent() to AttrTable. This returns the parent container for this AttrTable. I removed the old iostream methods from the library. These methods should not be used because in many cases since other parts of the library use the C/stdio functions for I/O. In older versions of gcc, it was possible to mix the two types of I/O systems, but not now (and not in other compilers). To change your code, look in the file Removed_functions.txt to see the functions/methods that have been removed. In all cases there is a version that takes a FILE * in place of the ostream &. Use the FILE* version. If you're performing I/O that relies on operator<<() to do type conversions, use an ostringstream in your code and then write the string to stdout (or wherever) using fprintf like: fprintf(stdout, "%s", oss.str().c_str()). Ugly, but easier in some cases than replacing lots of tested C++ I/O with fprintf calls. I removed old methods in Connect that have been deprecated for more than a year. See Removed_functions.txt I removed the old Pix methods. See removed_functions.txt. I removed the const char * overloads added to prevent collisions between the Pix and String (not string, but GNU's old libg String class) methods. Added protocol version number header to responses which use the set_mime...() functions. The new header is called "XDAP-Protocol" and its value is the two digit DAP protocol version number. This is now used by the client-side deserialize() methods; I assume that a server that does not announce its protocol version is a 2.0 server. Removed the set_mime...() functions which take an iostream; use the ones which take the FILE* instead. We replaced the iostream versions with FILE* versions a long time ago because the parsers all use FILE* I/O functions and mixing the C++ and C I/O is not predictable. The old functions were deprecated. To fix your code, just change the iostream variable to a FILE*. In most cases this will mean changing 'cout' to 'stdout.' News for Release 3.5.3 Changes to the Regex software. I've reimplemented the GNURegex code to use the only the POSIX functions. Because of this there are some subtle changes in the way the class Regex works. These changes address bugs that show up on Mac OS/X 10.4 (Tiger). 1) Meta characters like '{' now have to be escaped like '\\{' 2) See the docs for the Regex::match and Regex::serach methods. Regex:match returns the number of characters that match or -1 if there's no match. Regex::serach returns the position of the _first_ match (not the longest as with POSIX) and the length of that first match in the value-result parameter 'matchlen'. 3) The Regex constructor now takes only one argument, the regular expression to compile. There's a second ctor that takes a second parameter (an int) but it is a dummy. As a result of these changes, a small portion of the interface for libdap has changed. Build improvements from Patrice Dumas. Added a Mac OS/X package and RedHat rpm/srpm targets. There's also a pmsp file for use with Mac's PackageMaker. News for Release 3.5.2 Fixed a bug where malformed Error objects from servers caused an exception. This caused the original error message to be lost, not very helpful. The library used a compile-time switch to control use of the factory class for creation of objects at run-time. This was causing more trouble than it was preventing, so I removed it. Code should switch from the 'virtual constructors' to the factory class now. The Test classes in the subdir 'test' should now produce the same values on 64- and 32-bit machines. Unit tests should all work (although one of the tests for util.cc is known to fail on FC4). Revamped the build to use automake. Regression tests and the test classes are now in the subdir 'tests.' The unit tests are now in the subdir 'unit-tests' and are not built or run by default. Some of these tests require access to the Internet to work and I decided to make then not run using the make check target from the top level (the regression tests in 'tests' do run using the top-level check target) so that the build would work w/o access to the net. The rpm spec file has been updated. See INSTALL for information about building rpm distributions. Thanks to Patrice Dumas for help on the autoconf/make, with gnulib and the rpm spec file. News for Release 3.5.1 beta 2005/05/13 I changed some of the build parameters; the utility script is now named dap-config, the headers now install into $prefix/include/dap and the static library no longer has a version number appended. I've also written a rpm spec file which can be used to build a rpm file of/for a binary distribution. At this stage you'll need to be pretty savvy with RPM to get it to work; I'll write up instructions soon. News for Release 3.5.0 beta 2005/05/05 Changes in the way the software is organized: * First, the old 'DODS' CVS module is being broken up to facilitate more frequent releases of software. libdap++ has a new CVS module named 'libdap.' To access the software using CVS, use 'cvs co libdap' (where in the past you used 'cvs co DODS/src/dap' or 'cvs co DODS' and then changed into the DODS/src/dap directory). * The autoconf scripts have been updated; still no libtool or Makefile.am, but the scripts are much more robust. * The third-party packages are no longer bundled with the library. In a sense they never were, but they _were_ a part of the DODS CVS module. Now it's up to you to get and install the required packages. Look on the web site (or Google) for libxml2 and curl. We build using curl version 7.12.3 and libxml version 2.6.16; curl 7.12.0 and libxml2 2.5.7 should work. * The libdap software now installs in $prefix/{lib,include,bin} instead of inside the 'DODS tree.' By default $prefix is /usr/local; use the --prefix option of configure to specify a different directory than /usr/local. The library itself installs in $prefix/lib as libdap++.a.3.5; libdap++.a is symbolically linked to that file. The header files now install in $prefix/include/libdap-3.5; $prefix/include/libdap is symbolically linked to that directory. The utilities geturl and opendap-config have been renamed getdap and dap-config, respectively (the name geturl corresponds to another utility). The getdap utility is, except for the name change, exactly the same as geturl. The dap-config script provides a way to determine which libraries should be included when linking with a particular build of libdap++. It also provides information about the version of the library linked to $prefix/lib/libdap++.a and some other stuff. Use dap-config --help for a listing of all the options. Finally, the deflate program, which is used by some servers to provide compressed responses and is called by the library is not stored in $prefix/sbin (except on win32 where it's stored in $prefix/bin). The usage program is not currently installed; once we complete the reorganization process it will find a good home. Other changes to libdap++: * The library now uses a factory class to determine how to instantiate specializations of Byte, Int32, et cetera. The class BaseTypeFactory defines the interface for the factory and provides a default implementation of the class. This implementation instantiates Byte, ..., Grid. Also supplied with the library is TestTypeFactory which instantiates the Test type classes (TestByte, ..., TestGrid). If your use of the library requires that the type classes be specialized, then you must modify your software so that it includes a factory class that specializes BaseTypeFactory. Then, when you create a DDS (or DataDDS) pass a pointer to an instance of your factory to the DDS constructor (or use the new DDS::set_factory() method). Look at BaseTypeFactory and the example specialization TestTypeFactory. It's very straightforward to make the change. For applications which don't specialize the type classes, the default factory should be fine. To avoid using the new DDS constructor (which requires that a pointer to an instance of BaseTypeFactory be supplied), your code must #define the symbol DEFAULT_BASETYPE_FACTORY. If this symbol is not defined (at compile time) then various compile-time errors will occur (the idea being this will prevent software from building and silently ignoring specializations of the type classes). If defined, this use the default factory class. The documentation for the DDS constructor has some more information. * The library contains support for the DDX object. The DDX will become the foundation of the DAP3 protocol. It uses XML to describe the information currently represented using our 'curly-brace' notation and also bundles the attributes along with the variables they describe. This will simplify many processing tasks for clients. The software provided simplifies generating the DDX by building it from existing DAS and DDS objects. * There has also been some significant re-factoring in DDS and DAS: The DDS:send() method has been copied over to DODSFilter and the version in DDS has been deprecated and will be removed in a future version of the library. The library uses STL iterators almost exclusively and the next version will eliminate the ancient Pix class. * The function dods_root() is now called libdap_root() and tests the environment variable LIBDAP_ROOT. If that variable is not set, libdap_root() returns the value passed to the --prefix option of configure or /usr/local if the option was not used. Added to libdap++ is a function libdap_version() which returns the version number of the library. Note that libdap_version() is declared as extern "C" so that it can be used in configure tests to check for the library. * The servers no longer provide three programs to handle the das, dds and data requests. Instead one *_handler is provided. This reduces the size of the servers by a factor of three and paves the way toward integration of the HTML and ASCII code into the server binary, which will improve the performance and security of those features.