PostgreSQL Weekly News | PGWN - 29 janvier 2006

Par SAS le 05/02/2007 - 23:59

Nouvelles hebdomadaires de PostgreSQL - 28 janvier 2007

Nouvelle discussion en cours sur -hackers concernant les améliorations à apporter à autovacuum.

Andy Astor, CEO d'EnterpriseDB, a démenti avec vigueur toute politique éventuelle de récupération des adresses de courriel à partir des listes PostgreSQL à des fins commerciales.

Les nouveautés des produits dérivés

• Libpqxx 2.6.9 pre-release est sortie :
  http://thaiopensource.org/~jtv/tmp/pqxx/
• SQL/PSM 0.1 réalisé :
  http://postgresql.interweb.cz/index.php/SQL/PSM
• DBI-Link 2.0.0 réalisé :
  http://pgfoundry.org/projects/dbi-link/
• Another PostgreSQL Diff Tool 1.0.0_beta16 réalisé :
  http://pgfoundry.org/projects/apgdiff/
• PostgresDAC 2.3.7 réalisé :
  http://microolap.com/products/connectivity/postgresdac/download/
• Mammoth Replicator 8.0.10-1.7 réalisé :
  https://www.commandprompt.com/products/mammothreplicator/

Les jobs de janvier

http://archives.postgresql.org/pgsql-jobs/2007-01/threads.php

PostgreSQL Local

Le contributeur majeur Gavin Sherry et le leader de Postgres-R, Markus Schiltknecht, interviendront le 6 février à 7:00pm au meeting SFPUG à San Francisco au siège de Hyperic LLC. RSVP à http://postgresql.meetup.com/1/calendar/5380885/

PostgreSQLFr tient un stand lors de SolutionsLinux 2007, à Paris (France) du 30 janvier au 1er février. Jean-Paul Argudo, trésorier de l'association, conduira une session dédiée aux bases de données FLOSS (Free/Libre/OpenSource Software). Plus d'informations ici :
http://www.solutionslinux.fr/index.php

Il y aura un stand PostgreSQL au FOSDEM les 24 et 25 février à Bruxelles. La plupart des têtes pensantes des communautés PostgreSQL européennes y seront. Contacter de@postgresql.org pour en être.
http://www.fosdem.org/2007/

Le communauté PostgreSQL italienne organise une journée PostgreSQL cet été. Pour participer, suivre le lien http://www.pgday.it

PostgreSQL dans les média

• Planet PostgreSQL : http://www.planetpostgresql.org/
• General Bits, Archives et nouveaux articles : http://www.varlena.com/GeneralBits/

PostgreSQL Weekly News vous est présenté cette semaine par David Fetter et Devrim GUNDUZ.
Adaptation francophone de Stéphane Schildknecht.

Vos propositions d'informations doivent parvenir avant dimanche minuit à david@fetter.org.

Correctifs appliqués

Andrew Dunstan a commité :

• Fix plpython MSVC build in non-debug mode.
• Fix path problem in MSVC bison wrapper. per Joachim Wieland.
• In PL/Perl, remove unnecessary and now inaccurate cast which I should have removed with other old code.
• In PL/Perl, allow args to spi_prepare to be standard type aliaes as well as those known in pg_type. Fixes bug #2917. Add some regression tests for these cases.

Michael Meskes a commité :

• In ecpg, fixed expected files, so they are in sync with tests again.
• In ecpg, removed compiler warning due to unneeded unsigned declaration and removed regression test that triggers those libc precision bugs on some archs.

Neil Conway committed:

• Rename the uuid_t type to pg_uuid_t, to avoid a conflict with any definitions of uuid_t that may be provided by the system headers. This should hopefully fix the Win32 build problems reported by Magnus.
• Gevik Babakhani's patch which add a new builtin type, "uuid". This implements a UUID type, similar to that defined in RFC 4122. This patch includes the basic implementation, plus regression tests. Documentation and perhaps some additional functionality will come later. Catversion bumped. Review from Peter Eisentraut, Tom Lane, and Neil Conway.
• Magnus Hagander's patch which squelches some VC++ compiler warnings. Mark float literals with the "f" suffix, to distinguish them from doubles. Make some function declarations and definitions use the "const" qualifier for arguments consistently. Ignore warning 4102 ("unreferenced label"), because such warnings are always emitted by bison-generated code.
• Magnus Hagander's patch which adds a setlocal command to the beginning of build.bat. This is required to deal with buildenv.bat properly, so that PATH (for example) doesn't expand infintly. Per report from Joachim Wieland.
• Magnus Hagander's patch which adds a setlocal command to the beginning of build.bat. This is required to deal with buildenv.bat properly, so that PATH (for example) doesn't expand infintly. Per report from Joachim Wieland.

Peter Eisentraut a commité :

• In pgsql/src/backend/utils/adt/xml.c, reactivate libxml memory management via palloc, now that I think I've classified the conditions under which this is safe to do (see source code comment).
• In pgsql/src/backend/utils/adt/xml.c, add trailing zero byte in Unicode codepoint conversion.
• Various fixes in the logic of XML functions: Add new SQL command SET XML OPTION (also available via regular GUC) to control the DOCUMENT vs. CONTENT option in implicit parsing and serialization operations. Subtle corrections in the handling of the standalone property in xmlroot(). Allow xmlroot() to work on content fragments. Subtle corrections in the handling of the version property in xmlconcat(). Code refactoring for producing XML declarations.
• Simplify handling of XML error messages: Just use the string provided by libxml as the detail message. As per < http://archives.postgresql.org/pgsql-hackers/2006-12/msg01087.php>.
  For converting error codes to messages, we only need to cover those codes that we raise ourselves now.

Alvaro Herrera committed:

• Magnus Hagander's patch which is required for vcbuild to work after the changes to pg_proc.h (it's the same changes that are in gen_fmgrtab.sh in the unix build).

Tom Lane committed:

• Improve hash join to discard input tuples immediately if they can't match because they contain a null join key (and the join operator is known strict). Improves performance significantly when the inner relation contains a lot of nulls, as per bug #2930.
• Remove unnecessary checkpoint from PL regression tests. This was once handy to prevent core dump files from disappearing, but it's useless now because (a) we don't drop core in individual DB subdirectories anymore, and (b) CREATE DATABASE forces an internal checkpoint anyway.
• Remove some unnecessary conversion work in build_regtype_array().
• Repair oversight in creation of "append relations": we should set up rel->tuples as well as rel->rows, since some estimation functions expect both to be valid in every baserel. Per report from Dave Dutcher.
• Make some small improvements in the accuracy of plpgsql's error location reports; inspired by the misleading CONTEXT lines shown in recent bug report from Stefan Kaltenbrunner. Also, allow statement-type names shown in these messages to be translated.
• Fix up plpgsql's "simple expression" evaluation mechanism so that it behaves safely in the presence of subtransactions. To ensure that any ExprContext shutdown callbacks are called at the right times, we have to have a separate EState for each level of subtransaction. Per "TupleDesc reference leak" bug report from Stefan Kaltenbrunner. Although I'm convinced the code is wrong as far back as 8.0, it doesn't seem that there are any ways for the problem to really manifest before 8.2: AFAICS, 8.0 and 8.1 only use the ExprContextCallback mechanism to handle set-returning functions, which cannot usefully be executed in a "simple expression" anyway. Hence, no backpatch before 8.2 --- the risk of unforeseen breakage seems to outweigh the chance of fixing something.
• Drat, can't fit an additional argument into log_error. Is it worth an sprintf pushup to be sure we can report something useful for out-of-range exitstatus?
• Clean up broken usage of HAVE_DECL_SYS_SIGLIST and inconsistent/poorly formatted error messages.
• Add a delay at the start of the stats test, to let any prior stats activity quiesce. Possibly this will fix the large increase in non-reproducible stats test failures we've noted since turning on stats_row_level by default.
• Dept of second thoughts: the IQ of estimate_array_length() needs to be kept on par with that of scalararraysel(), else estimates that should track might not. Hence teach it about binary-compatible cases, too.
• Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
• Correct an old logic error in btree page splitting: when considering a split exactly at the point where we need to insert a new item, the calculation used the wrong size for the "high key" of the new left page. This could lead to choosing an unworkable split, resulting in "PANIC: failed to add item to the left sibling" (or "right sibling") failure. Although this bug has been there a long time, it's very difficult to trigger a failure before 8.2, since there was generally a lot of free space on both sides of a chosen split. In 8.2, where the user-selected fill factor determines how much free space the code tries to leave, an unworkable split is much more likely. Report by Joe Conway, diagnosis and fix by Heikki Linnakangas.
• Back-port changes of Jan 16 and 17 to "revoke" pending fsync requests during DROP TABLE and DROP DATABASE. Should prevent unexpected "permission denied" failures on Windows, and is cleaner on other platforms too since we no longer have to take it on faith that ENOENT is okay during an fsync attempt. Patched as far back as 8.1; per recent discussion I think we are not going to worry about Windows-specific issues in 8.0 anymore.
• On Windows, use pgwin32_waitforsinglesocket() instead of select() to wait for input in the stats collector. Our select() emulation is apparently buggy for UDP sockets :-(. This should resolve problems with stats collection (and hence autovacuum) failing under more than minimal load. Diagnosis and patch by Magnus Hagander.
• Get pg_utf_mblen(), pg_utf2wchar_with_len(), and utf2ucs() all on the same page about the maximum UTF8 sequence length we support (4 bytes since 8.1, 3 before that). pg_utf2wchar_with_len never got updated to support 4-byte characters at all, and in any case had a buffer-overrun risk in that it could produce multiple pg_wchars from what mblen claims to be just one UTF8 character. The only reason we don't have a major security hole is that most callers allocate worst-case output buffers; the sole exception in released versions appears to be pre-8.2 iwchareq() (ie, ILIKE), which can be crashed due to zeroing out its return address --- but AFAICS that can't be exploited for anything more than a crash, due to inability to control what gets written there. Per report from James Russell and Michael Fuhr. Pre-8.1 the risk is much less, but I still think pg_utf2wchar_with_len's behavior given an incomplete final character risks buffer overrun, so back-patch that logic change anyway. This patch also makes sure that UTF8 sequences exceeding the supported length (whichever it is) are consistently treated as error cases, rather than being treated like a valid shorter sequence in some places.
• Relax an Assert() that has been found to be too strict in some situations involving unions of types having typmods. Variants of the failure are known to occur in 8.1 and up; not sure if it's possible in 8.0 and 7.4, but since the code exists that far back, I'll just patch 'em all. Per report from Brian Hurt.
• Update xindex.sgml to discuss operator families.
• Update pg_dump to support dumping operator families.
• Add CREATE/ALTER/DROP OPERATOR FAMILY commands, also COMMENT ON OPERATOR FAMILY; and add FAMILY option to CREATE OPERATOR CLASS to allow adding a class to a pre-existing family. Per previous discussion. Man, what a tedious lot of cutting and pasting ...
• Put back planner's ability to cache the results of mergejoinscansel(), which I had removed in the first cut of the EquivalenceClass rewrite to simplify that patch a little. But it's still important --- in a four-way join problem mergejoinscansel() was eating about 40% of the planning time according to gprof. Also, improve the EquivalenceClass code to re-use join RestrictInfos rather than generating fresh ones for each join considered. This saves some memory space but more importantly improves the effectiveness of caching planning info in RestrictInfos.

Bruce Momjian a commité :

• Use autoconf build-in sys_siglist macro AC_DECL_SYS_SIGLIST, rather than create our own.
• Add signal.h for sys_siglist reference.
• Use sys_siglist[] to print out signal names for signal exits, rather than just numbers.
• Modify SGML makefile to allow 'gmake draft' to build draft html, so 'draft' is now a target and no longer a modifier.
• Add to TODO: "Enforce typmod for function inputs, function results and parameters for spi_prepare'd statements called from PLs."
• Add to TODO: "Consider having the background writer update the transaction status hint bits before writing out the page."
• Add to TODO: "Consider increasing NUM_CLOG_BUFFERS."
• Update installation wording for an upgrade to state that dump/restore should not be done, per Peter.
• Control openjade draft-mode by variable DRAFT, rather than whether the version tag is 'devel'.
• Update wording of installation when upgrading, to distinguish major release upgrades more clearly from minor ones.
• Modify draft SGML instructions to use DRAFT=Y because recursion was causing html to be called twice --- no way to exit the makefile after the recursion returns.
• Update BSD/OS platform for 8.2.
• In TODO, add URL for shrinking tuple to just its headers.
• Update Win32 exception comment.
• Update pg_dumpall -f option description.
• In TODO, mark as done, "Add a GUC variable to control the tablespace for temporary objects pg_dumpall" per Albert Cervera Areny's patch.
• Albert Cervera Areny's patch with documentation by Jaime Casanova to add GUC temp_tablespaces to provide a default location for temporary objects.
• Properly detoast access to bytea field pg_trigger.tgargs. Old code might cause server crash. Backpatch to 8.2.X.
• Dave Page's patch which adds 'output file' option for pg_dumpall, especially useful for Win32, where output redirection of child processes (pg_dump) doesn't work.
• Remove developers list from TODO list now that we have URLs to reference discussions.
• In TODO, mark as done, "Add -f to pg_dumpall" per Dave Page's patch.
• Dave Page patch which allows pg_dumpall to specify a database name rather than the default 'template1'.
• Dave Page's patch which adds --tablespaces-only and --roles-only options to pg_dumpall.
• Simon Riggs's patch to prevent WAL logging when COPY is done in the same transaction that created it.
• Magnus Hagander's patch that disables the building of ecpg when using MSVC if pthreads is not specified.
• In pgsql/src/include/port/win32.h, add comment about exception lists in both winnt.h and ntstatus.h.
• Remove newline from error message because URL is gone.
• Back out use of FormatMessage(), does error values, not exception values. Point to /include/ntstatus.h for an exception list, rather than a URL.
• Print meaningfull error text for abonormal process exit on Win32, rather than hex codes, using FormatMessage().
• In FAQ, update Bruce Momjian's email address.
• Use errhint() for WIN32 SIGTERM message, where possible.
• In pgsql/src/include/port/win32.h, put hex value in upper case.
• When system() fails in Win32, report it as an exception, print the exception value in hex, and give a URL where the value can be looked-up.

Correctifs rejetés (à ce jour)

Guido Goldstein's patch to fix PL/PythonU functions; return true/false for boolean, rather than 1/0. This helps when creating trigger functions that output SQL. It also broke most machines on the buildfarm because it only works with Python 2.3 or higher.

Magnus Hagander's first version of his patch to clean up warnings generated by VC++. It was too large and invasive, and had some thinkos. He is working on the next version.

Correctifs en attente

• Arul Shaji sent in another version of his implementation of pg_get_domaindef ;
• Gevik Babakhani sent in another version of his uuid patch. Peter Eisentraut gave some feedback. Gevik sent in another version per the comments and added tests ;
• Jeremy Drake sent in a couple of versions of a patch which allows unprivileged users to CREATE LANGUAGE for trusted languages already present in pg_pltemplate ;
• Teodor Sigaev sent in a doc patch which updates the docbook instructions for FreeBSD, and another which adds a table of pairwise compatibilities between the different lock modes ;
• Andrew Dunstan sent in a patch which lets people use the aliases in pg_type for parameters in PL/Tcl and PL/PythonU, e.g. integer instead of int4 ;
• Pavan Deolasee sent in a patch which should marginally improve the ctid chain following code path when current and the next tuple in the chain are in the same block ;
• Alvaro Herrera sent in a patch which separates autovacuum in two families of processes: one is the "launcher", in charge of examining statistics and deciding when to start a worker. The other is the worker, which is started by the postmaster under command of the launcher, and processes what the launcher tells it to process (by way of setting info up in shared memory) ;
• Pavel Stehule sent in a patch to add ANSI SQL scrollable cursor support to SPI and PL/PgSQL ;
• Jeremy Drake sent in another version of his patch which lets you install PLs without being superuser.