after vacuum analyze, explain still wrong

Started by Sim Zacksalmost 17 years ago8 messagesgeneral
Jump to latest
#1Sim Zacks
sim@compulab.co.il

I ran vacuum analyze and immediately after I ran my query and the
estimated rows are way off. I suspect that it is something in my
configuration, but I don't know what.
I pasted my postgresql.conf file under the explain results.

Thank you
Sim

GroupAggregate (cost=4542.87..4543.12 rows=1 width=32) (actual
time=150947.014..150984.246 rows=83 loops=1)
-> GroupAggregate (cost=4542.87..4543.03 rows=1 width=120) (actual
time=150946.978..150975.130 rows=3754 loops=1)
-> Sort (cost=4542.87..4542.88 rows=3 width=120) (actual
time=150946.949..150950.724 rows=7123 loops=1)
Sort Key: a.assembliesbatchid, a.duedate, a.assemblyid,
a.assemblyname, a.ownerid, a.partid, a.quantity, a.stock, a.prioruse,
a.units, a.qtyperunit, a.priorusebatch
-> Nested Loop Left Join (cost=3634.74..4542.85 rows=3
width=120) (actual time=47753.176..150896.984 rows=7123 loops=1)
Join Filter: (e.partid = a.partid)
-> GroupAggregate (cost=2679.49..2679.56 rows=1
width=89) (actual time=47702.211..47873.649 rows=3754 loops=1)
-> Sort (cost=2679.49..2679.50 rows=1
width=89) (actual time=47702.193..47755.292 rows=41197 loops=1)
Sort Key: a.assembliesbatchid,
a.duedate, a.assemblyid, a.assemblyname, a.ownerid, a.partid,
a.quantity, a.stock, a.units, a.qtyperunit, a.leadfree
-> Nested Loop Left Join
(cost=2556.68..2679.48 rows=1 width=89) (actual time=9767.220..46933.103
rows=41197 loops=1)
Join Filter: ((a.partid =
b.partid) AND ((a.ownerid IS NULL) OR (a.ownerid = 1)) AND (a.leadfree =
q.leadfree) AND ((a.duedate > c.duedate) OR ((a.duedate = c.duedate) AND
(a.assembliesbatchid > c.assembliesbatchid))))
-> GroupAggregate
(cost=2381.63..2381.68 rows=1 width=82) (actual time=9754.658..9773.515
rows=3754 loops=1)
-> Sort
(cost=2381.63..2381.64 rows=1 width=82) (actual time=9754.641..9757.533
rows=3754 loops=1)
Sort Key:
d.assembliesbatchid, d.duedate, a.assemblyid, a.assemblyname, c.ownerid,
e.partid, COALESCE(c.stock, 0::bigint), d.units, e.quantity, a.leadfree
-> Nested Loop Left
Join (cost=145.04..2381.62 rows=1 width=82) (actual
time=39.045..9726.318 rows=3754 loops=1)
-> Nested Loop
Left Join (cost=145.04..2381.33 rows=1 width=74) (actual
time=38.955..9678.057 rows=3754 loops=1)
Join
Filter: ((c.partid = e.partid) AND leadcompcheck_ab(a.leadfree,
c.leadstateid))
-> Nested
Loop (cost=145.04..2366.26 rows=1 width=62) (actual time=0.791..83.482
rows=3754 loops=1)
->
Nested Loop Left Join (cost=145.04..2365.59 rows=1 width=28) (actual
time=0.721..52.840 rows=3754 loops=1)

Filter: (f.commited IS NOT TRUE)

-> Hash Join (cost=136.62..273.55 rows=168 width=32) (actual
time=0.706..14.201 rows=3754 loops=1)

Hash Cond: (e.assembliesbatchid = d.assembliesbatchid)

-> Seq Scan on allocatedassemblies e (cost=0.00..121.45
rows=3679 width=12) (actual time=0.039..6.140 rows=3754 loops=1)

Filter: ((- quantity) <> 0)

-> Hash (cost=135.51..135.51 rows=89 width=24) (actual
time=0.612..0.612 rows=88 loops=1)

-> Bitmap Heap Scan on assembliesbatch d
(cost=17.69..135.51 rows=89 width=24) (actual time=0.122..0.521 rows=88
loops=1)

Recheck Cond: (assembliesbatchstatusid = ANY
('{1,2,4,7}'::integer[]))

-> Bitmap Index Scan on
fki_assembliesbatch_assembliesbatchstatus_id (cost=0.00..17.66 rows=89
width=0) (actual time=0.098..0.098 rows=89 loops=1)

Index Cond: (assembliesbatchstatusid = ANY
('{1,2,4,7}'::integer[]))

-> Bitmap Heap Scan on stocklog f (cost=8.43..12.44 rows=1 width=9)
(actual time=0.006..0.006 rows=0 loops=3754)

Recheck Cond: ((f.refid = d.batchid) AND (f.partid = e.partid))

Filter: (transtypeid = 3)

-> BitmapAnd (cost=8.43..8.43 rows=1 width=0) (actual
time=0.004..0.004 rows=0 loops=3754)

-> Bitmap Index Scan on referenceidsl (cost=0.00..3.62
rows=29 width=0) (actual time=0.001..0.001 rows=1 loops=3754)

Index Cond: (f.refid = d.batchid)

-> Bitmap Index Scan on ix_partsstocklog
(cost=0.00..4.54 rows=132 width=0) (actual time=0.116..0.116 rows=379
loops=67)

Index Cond: (f.partid = e.partid)
->
Index Scan using assemblies_pkey on assemblies a (cost=0.00..0.65
rows=1 width=38) (actual time=0.004..0.005 rows=1 loops=3754)

Index Cond: (d.assemblyid = a.assemblyid)
->
Function Scan on stockperowner_lead_ab c (cost=0.00..15.00 rows=5
width=20) (actual time=0.010..1.451 rows=1965 loops=3754)

Filter: (ownerid = 1)
-> Index Scan
using idx_u_assidpartid on partsassembly b (cost=0.00..0.28 rows=1
width=16) (actual time=0.008..0.009 rows=1 loops=3754)
Index
Cond: ((e.partid = b.partid) AND (b.assemblyid = a.assemblyid))
-> Hash Join
(cost=175.05..293.57 rows=169 width=25) (actual time=0.005..7.779
rows=3755 loops=3754)
Hash Cond:
(b.assembliesbatchid = c.assembliesbatchid)
-> Seq Scan on
allocatedassemblies b (cost=0.00..102.97 rows=3697 width=12) (actual
time=0.003..2.238 rows=3755 loops=3754)
-> Hash
(cost=173.93..173.93 rows=89 width=17) (actual time=2.200..2.200 rows=88
loops=1)
-> Hash Join
(cost=54.44..173.93 rows=89 width=17) (actual time=1.632..2.116 rows=88
loops=1)
Hash Cond:
(c.assemblyid = q.assemblyid)
-> Bitmap Heap
Scan on assembliesbatch c (cost=21.78..140.04 rows=89 width=20) (actual
time=0.114..0.441 rows=88 loops=1)
Recheck
Cond: ((assembliesbatchstatusid = 1) OR (assembliesbatchstatusid = 2) OR
(assembliesbatchstatusid = 4) OR (assembliesbatchstatusid = 7))
->
BitmapOr (cost=21.78..21.78 rows=89 width=0) (actual time=0.093..0.093
rows=0 loops=1)
->
Bitmap Index Scan on fki_assembliesbatch_assembliesbatchstatus_id
(cost=0.00..8.86 rows=81 width=0) (actual time=0.071..0.071 rows=81 loops=1)

Index Cond: (assembliesbatchstatusid = 1)
->
Bitmap Index Scan on fki_assembliesbatch_assembliesbatchstatus_id
(cost=0.00..4.26 rows=1 width=0) (actual time=0.003..0.003 rows=0 loops=1)

Index Cond: (assembliesbatchstatusid = 2)
->
Bitmap Index Scan on fki_assembliesbatch_assembliesbatchstatus_id
(cost=0.00..4.26 rows=1 width=0) (actual time=0.003..0.003 rows=0 loops=1)

Index Cond: (assembliesbatchstatusid = 4)
->
Bitmap Index Scan on fki_assembliesbatch_assembliesbatchstatus_id
(cost=0.00..4.31 rows=8 width=0) (actual time=0.014..0.014 rows=8 loops=1)

Index Cond: (assembliesbatchstatusid = 7)
-> Hash
(cost=20.63..20.63 rows=963 width=5) (actual time=1.506..1.506 rows=963
loops=1)
-> Seq
Scan on assemblies q (cost=0.00..20.63 rows=963 width=5) (actual
time=0.009..0.768 rows=963 loops=1)
-> Hash Join (cost=955.25..1802.66 rows=4850
width=28) (actual time=0.046..27.056 rows=669 loops=3754)
Hash Cond: (e.pnid = g.pnid)
-> Hash Join (cost=615.50..1353.75 rows=4853
width=32) (actual time=0.040..25.829 rows=669 loops=3754)
Hash Cond: (e.poid = f.poid)
-> Seq Scan on poparts e
(cost=0.00..583.48 rows=21248 width=32) (actual time=0.007..12.436
rows=21248 loops=3754)
-> Hash (cost=590.41..590.41 rows=2007
width=8) (actual time=7.297..7.297 rows=194 loops=1)
-> Seq Scan on pos f
(cost=0.00..590.41 rows=2007 width=8) (actual time=0.030..7.123 rows=194
loops=1)
Filter: ((postatusid >= 20)
AND (postatusid <= 59) AND (NOT isrfq))
-> Hash (cost=234.33..234.33 rows=8433
width=4) (actual time=13.265..13.265 rows=8433 loops=1)
-> Seq Scan on manufacturerpartpn g
(cost=0.00..234.33 rows=8433 width=4) (actual time=0.025..7.065
rows=8433 loops=1)
Total runtime: 150986.997 ms

# -----------------------------
# PostgreSQL configuration file
# -----------------------------
#
# This file consists of lines of the form:
#
# name = value
#
# (The '=' is optional.) White space may be used. Comments are introduced
# with '#' anywhere on a line. The complete list of option names and
# allowed values can be found in the PostgreSQL documentation. The
# commented-out settings shown in this file represent the default values.
#
# Please note that re-commenting a setting is NOT sufficient to revert it
# to the default value, unless you restart the server.
#
# Any option can also be given as a command line switch to the server,
# e.g., 'postgres -c log_connections=on'. Some options can be changed at
# run-time with the 'SET' SQL command.
#
# This file is read on server startup and when the server receives a
# SIGHUP. If you edit the file on a running system, you have to SIGHUP the
# server for the changes to take effect, or use "pg_ctl reload". Some
# settings, which are marked below, require a server shutdown and restart
# to take effect.
#
# Memory units: kB = kilobytes MB = megabytes GB = gigabytes
# Time units: ms = milliseconds s = seconds min = minutes h = hours d
= days

#---------------------------------------------------------------------------
# FILE LOCATIONS
#---------------------------------------------------------------------------

# The default values of these variables are driven from the -D command line
# switch or PGDATA environment variable, represented here as ConfigDir.

#data_directory = 'ConfigDir' # use data in another directory
# (change requires restart)
#hba_file = 'ConfigDir/pg_hba.conf' # host-based authentication file
# (change requires restart)
#ident_file = 'ConfigDir/pg_ident.conf' # ident configuration file
# (change requires restart)

# If external_pid_file is not explicitly set, no extra PID file is written.
#external_pid_file = '(none)' # write an extra PID file
# (change requires restart)

#---------------------------------------------------------------------------
# CONNECTIONS AND AUTHENTICATION
#---------------------------------------------------------------------------

# - Connection Settings -

#listen_addresses = 'localhost' # what IP address(es) to listen on;
listen_addresses='*' # comma-separated list of addresses;
# defaults to 'localhost', '*' = all
# (change requires restart)
#port = 5432 # (change requires restart)
max_connections = 300 # (change requires restart)
# Note: increasing max_connections costs ~400 bytes of shared memory per
# connection slot, plus lock space (see max_locks_per_transaction). You
# might also need to raise shared_buffers to support more connections.
#superuser_reserved_connections = 3 # (change requires restart)
#unix_socket_directory = '' # (change requires restart)
#unix_socket_group = '' # (change requires restart)
#unix_socket_permissions = 0777 # octal
# (change requires restart)
#bonjour_name = '' # defaults to the computer name
# (change requires restart)

# - Security & Authentication -

#authentication_timeout = 1min # 1s-600s
#ssl = off # (change requires restart)
#password_encryption = on
#db_user_namespace = off

# Kerberos
#krb_server_keyfile = '' # (change requires restart)
#krb_srvname = 'postgres' # (change requires restart)
#krb_server_hostname = '' # empty string matches any keytab entry
# (change requires restart)
#krb_caseins_users = off # (change requires restart)

# - TCP Keepalives -
# see 'man 7 tcp' for details

#tcp_keepalives_idle = 0 # TCP_KEEPIDLE, in seconds;
# 0 selects the system default
#tcp_keepalives_interval = 0 # TCP_KEEPINTVL, in seconds;
# 0 selects the system default
#tcp_keepalives_count = 0 # TCP_KEEPCNT;
# 0 selects the system default

#---------------------------------------------------------------------------
# RESOURCE USAGE (except WAL)
#---------------------------------------------------------------------------

# - Memory -

shared_buffers = 32MB # min 128kB or max_connections*16kB
# (change requires restart)
temp_buffers = 2MB # min 800kB
#max_prepared_transactions = 5 # can be 0 or more
# (change requires restart)
# Note: increasing max_prepared_transactions costs ~600 bytes of shared
memory
# per transaction slot, plus lock space (see max_locks_per_transaction).
work_mem = 2MB # min 64kB
maintenance_work_mem = 16MB
max_stack_depth = 2MB # min 100kB

# - Free Space Map -

max_fsm_pages = 50000 # min max_fsm_relations*16, 6 bytes each
# (change requires restart)
max_fsm_relations = 1000 # min 100, ~70 bytes each
# (change requires restart)

# - Kernel Resource Usage -

#max_files_per_process = 1000 # min 25
# (change requires restart)
#shared_preload_libraries = '' # (change requires restart)

# - Cost-Based Vacuum Delay -

#vacuum_cost_delay = 0 # 0-1000 milliseconds
#vacuum_cost_page_hit = 1 # 0-10000 credits
#vacuum_cost_page_miss = 10 # 0-10000 credits
#vacuum_cost_page_dirty = 20 # 0-10000 credits
#vacuum_cost_limit = 200 # 0-10000 credits

# - Background writer -

#bgwriter_delay = 200ms # 10-10000ms between rounds
#bgwriter_lru_percent = 1.0 # 0-100% of LRU buffers scanned/round
#bgwriter_lru_maxpages = 5 # 0-1000 buffers max written/round
#bgwriter_all_percent = 0.333 # 0-100% of all buffers scanned/round
#bgwriter_all_maxpages = 5 # 0-1000 buffers max written/round

#---------------------------------------------------------------------------
# WRITE AHEAD LOG
#---------------------------------------------------------------------------

# - Settings -

#fsync = on # turns forced synchronization on or off
#wal_sync_method = fsync # the default is the first option
# supported by the operating system:
# open_datasync
# fdatasync
# fsync
# fsync_writethrough
# open_sync
#full_page_writes = on # recover from partial page writes
#wal_buffers = 64kB # min 32kB
# (change requires restart)
#commit_delay = 0 # range 0-100000, in microseconds
#commit_siblings = 5 # range 1-1000

# - Checkpoints -

#checkpoint_segments = 3 # in logfile segments, min 1, 16MB each
#checkpoint_timeout = 5min # range 30s-1h
#checkpoint_warning = 30s # 0 is off

# - Archiving -

#archive_command = '' # command to use to archive a logfile segment
#archive_timeout = 0 # force a logfile segment switch after this
# many seconds; 0 is off

#---------------------------------------------------------------------------
# QUERY TUNING
#---------------------------------------------------------------------------

# - Planner Method Configuration -

#enable_bitmapscan = on
#enable_hashagg = on
#enable_hashjoin = on
#enable_indexscan = on
#enable_mergejoin = on
#enable_nestloop = on
#enable_seqscan = on
#enable_sort = on
#enable_tidscan = on

# - Planner Cost Constants -

#seq_page_cost = 1.0 # measured on an arbitrary scale
#random_page_cost = 4.0 # same scale as above
#cpu_tuple_cost = 0.01 # same scale as above
#cpu_index_tuple_cost = 0.005 # same scale as above
#cpu_operator_cost = 0.0025 # same scale as above
effective_cache_size = 150MB

# - Genetic Query Optimizer -

#geqo = on
#geqo_threshold = 12
#geqo_effort = 5 # range 1-10
#geqo_pool_size = 0 # selects default based on effort
#geqo_generations = 0 # selects default based on effort
#geqo_selection_bias = 2.0 # range 1.5-2.0

# - Other Planner Options -

#default_statistics_target = 10 # range 1-1000
#constraint_exclusion = off
#from_collapse_limit = 8
#join_collapse_limit = 8 # 1 disables collapsing of explicit
# JOINs

#---------------------------------------------------------------------------
# ERROR REPORTING AND LOGGING
#---------------------------------------------------------------------------

# - Where to Log -

#log_destination = 'stderr' # Valid values are combinations of
log_destination = 'syslog' # stderr, syslog and eventlog,
# depending on platform.

# This is used when logging to stderr:
#redirect_stderr = off # Enable capturing of stderr into log
# files
# (change requires restart)

# These are only used if redirect_stderr is on:
#log_directory = 'pg_log' # Directory where log files are written
# Can be absolute or relative to PGDATA
#log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # Log file name pattern.
# Can include strftime() escapes
#log_truncate_on_rotation = off # If on, any existing log file of the same
# name as the new log file will be
# truncated rather than appended to. But
# such truncation only occurs on
# time-driven rotation, not on restarts
# or size-driven rotation. Default is
# off, meaning append to existing files
# in all cases.
#log_rotation_age = 1d # Automatic rotation of logfiles will
# happen after that time. 0 to
# disable.
#log_rotation_size = 10MB # Automatic rotation of logfiles will
# happen after that much log
# output. 0 to disable.

# These are relevant when logging to syslog:
#syslog_facility = 'LOCAL0'
#syslog_ident = 'postgres'

# - When to Log -

#client_min_messages = notice # Values, in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# log
# notice
# warning
# error

#log_min_messages = notice # Values, in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic

#log_error_verbosity = default # terse, default, or verbose messages

#log_min_error_statement = error # Values in order of increasing severity:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# fatal
# panic (effectively off)

#log_min_duration_statement = -1 # -1 is disabled, 0 logs all statements
# and their durations.

#silent_mode = off # DO NOT USE without syslog or
# redirect_stderr
# (change requires restart)

# - What to Log -

#debug_print_parse = off
#debug_print_rewritten = off
#debug_print_plan = off
#debug_pretty_print = off
#log_connections = off
#log_disconnections = off
#log_duration = off
#log_line_prefix = '' # Special values:
# %u = user name
# %d = database name
# %r = remote host and port
# %h = remote host
# %p = PID
# %t = timestamp (no milliseconds)
# %m = timestamp with milliseconds
# %i = command tag
# %c = session id
# %l = session line number
# %s = session start timestamp
# %x = transaction id
# %q = stop here in non-session
# processes
# %% = '%'
# e.g. '<%u%%%d> '
#log_statement = 'none' # none, ddl, mod, all
#log_hostname = off

#---------------------------------------------------------------------------
# RUNTIME STATISTICS
#---------------------------------------------------------------------------

# - Query/Index Statistics Collector -

#stats_command_string = on
#update_process_title = on

stats_start_collector = on # needed for block or row stats
# (change requires restart)
#stats_block_level = off
#stats_row_level = off
stats_row_level = on
stats_reset_on_server_start = off # (change requires restart)

# - Statistics Monitoring -

#log_parser_stats = off
#log_planner_stats = off
#log_executor_stats = off
#log_statement_stats = off

#---------------------------------------------------------------------------
# AUTOVACUUM PARAMETERS
#---------------------------------------------------------------------------

#autovacuum = off # enable autovacuum subprocess?
autovacuum = on # 'on' requires stats_start_collector
# and stats_row_level to also be on
#autovacuum_naptime = 1min # time between autovacuum runs
autovacuum_vacuum_threshold = 50 # min # of tuple updates before
# vacuum
autovacuum_analyze_threshold = 75 # min # of tuple updates before
# analyze
autovacuum_vacuum_scale_factor = 0.1 # fraction of rel size before
# vacuum
autovacuum_analyze_scale_factor = 0.05 # fraction of rel size before
# analyze
#autovacuum_freeze_max_age = 200000000 # maximum XID age before forced
vacuum
# (change requires restart)
#autovacuum_vacuum_cost_delay = -1 # default vacuum cost delay for
# autovacuum, -1 means use
# vacuum_cost_delay
#autovacuum_vacuum_cost_limit = -1 # default vacuum cost limit for
# autovacuum, -1 means use
# vacuum_cost_limit

#---------------------------------------------------------------------------
# CLIENT CONNECTION DEFAULTS
#---------------------------------------------------------------------------

# - Statement Behavior -

search_path = '"$user",public,sales' # schema names
#default_tablespace = '' # a tablespace name, '' uses
# the default
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
#statement_timeout = 0 # 0 is disabled
#vacuum_freeze_min_age = 100000000

# - Locale and Formatting -

datestyle = 'iso, mdy'
#timezone = unknown # actually, defaults to TZ
# environment setting
#timezone_abbreviations = 'Default' # select the set of available
timezone
# abbreviations. Currently, there are
# Default
# Australia
# India
# However you can also create your own
# file in share/timezonesets/.
#extra_float_digits = 0 # min -15, max 2
#client_encoding = sql_ascii # actually, defaults to database
# encoding

# These settings are initialized by initdb -- they might be changed
lc_messages = 'C' # locale for system error message
# strings
lc_monetary = 'C' # locale for monetary formatting
lc_numeric = 'C' # locale for number formatting
lc_time = 'C' # locale for time formatting

# - Other Defaults -

#explain_pretty_print = on
#dynamic_library_path = '$libdir'
#local_preload_libraries = ''

#---------------------------------------------------------------------------
# LOCK MANAGEMENT
#---------------------------------------------------------------------------

#deadlock_timeout = 1s
#max_locks_per_transaction = 64 # min 10
# (change requires restart)
# Note: each lock table slot uses ~270 bytes of shared memory, and there are
# max_locks_per_transaction * (max_connections + max_prepared_transactions)
# lock table slots.

#---------------------------------------------------------------------------
# VERSION/PLATFORM COMPATIBILITY
#---------------------------------------------------------------------------

# - Previous Postgres Versions -

add_missing_from = on
#array_nulls = on
#backslash_quote = safe_encoding # on, off, or safe_encoding
#default_with_oids = off
escape_string_warning = on
#standard_conforming_strings = off
#regex_flavor = advanced # advanced, extended, or basic
#sql_inheritance = on

# - Other Platforms & Clients -

#transform_null_equals = off

#---------------------------------------------------------------------------
# CUSTOMIZED OPTIONS
#---------------------------------------------------------------------------

#custom_variable_classes = '' # list of custom variable class names

#2Sim Zacks
sim@compulab.co.il
In reply to: Sim Zacks (#1)
Re: after vacuum analyze, explain still wrong

increase default_statistics_target to 100 for instance, and retry.

I changed the setting, reloaded the configuration (without restarting
the server) ran vacuum analyze and the results were the exact same.

Any other thoughts?

Sim

#3Grzegorz Jaśkiewicz
gryzman@gmail.com
In reply to: Sim Zacks (#2)
Re: after vacuum analyze, explain still wrong

give us postgresql version as well, maybe show query and at least table layout.

#4Sim Zacks
sim@compulab.co.il
In reply to: Grzegorz Jaśkiewicz (#3)
Re: after vacuum analyze, explain still wrong

Grzegorz Jaśkiewicz wrote:

give us postgresql version as well, maybe show query and at least table layout.

"PostgreSQL 8.2.4 on i386-pc-linux-gnu, compiled by GCC
i386-pc-linux-gnu-gcc (GCC) 4.1.1 (Gentoo 4.1.1)"

It is a view based on 4 layers of views with a lot of tables and
functions thrown in.

#5Sim Zacks
sim@compulab.co.il
In reply to: Sim Zacks (#4)
Re: after vacuum analyze, explain still wrong

Sim Zacks wrote:

Grzegorz Jaśkiewicz wrote:

give us postgresql version as well, maybe show query and at least table layout.

The queries look like (The one I'm trying to run is the last one, which
is based on all the previous ones).:

CREATE OR REPLACE VIEW assembliesstockbatch AS
SELECT d.assembliesbatchid, d.duedate, a.assemblyid, a.assemblyname,
c.ownerid, e.partid, d.units, - e.quantity AS quantity,
COALESCE(c.stock, 0::bigint) AS stock, max(b.quantity) AS qtyperunit,
a.leadfree
FROM assemblies a
JOIN assembliesbatch d ON d.assemblyid = a.assemblyid
JOIN allocatedassemblies e ON e.assembliesbatchid = d.assembliesbatchid
LEFT JOIN partsassembly b ON b.assemblyid = a.assemblyid AND e.partid
= b.partid
LEFT JOIN stockperowner_lead_ab() c(partid, ownerid, stock,
leadstateid) ON c.partid = e.partid AND c.ownerid = 1 AND
leadcompcheck_ab(a.leadfree, c.leadstateid)
LEFT JOIN stocklog f ON f.refid = d.batchid AND f.transtypeid = 3 AND
f.partid = e.partid
WHERE (d.assembliesbatchstatusid = ANY (ARRAY[1, 2, 4, 7])) AND
f.commited IS NOT TRUE
GROUP BY d.assembliesbatchid, d.duedate, a.assemblyid, a.assemblyname,
c.ownerid, e.partid, COALESCE(c.stock, 0::bigint), d.units, e.quantity,
a.leadfree;

CREATE OR REPLACE VIEW assembliesstockbatchprioruse AS
SELECT a.assembliesbatchid, a.duedate, a.assemblyid, a.assemblyname,
a.ownerid, a.partid, a.units, a.quantity, a.stock, COALESCE(sum(-
b.quantity)::double precision, 0::double precision) AS prioruse, sum(
CASE
WHEN c.batchid IS NOT NULL THEN - b.quantity
ELSE 0
END)::double precision AS priorusebatch, a.qtyperunit, a.leadfree
FROM assembliesstockbatch a
LEFT JOIN (allocatedassemblies b
JOIN assembliesbatch c ON b.assembliesbatchid = c.assembliesbatchid
AND (c.assembliesbatchstatusid = 1 OR c.assembliesbatchstatusid = 2 OR
c.assembliesbatchstatusid = 4 OR c.assembliesbatchstatusid = 7)
JOIN assemblies q ON q.assemblyid = c.assemblyid) ON a.partid =
b.partid AND (a.ownerid IS NULL OR a.ownerid = 1) AND a.leadfree =
q.leadfree AND (a.duedate > c.duedate OR a.duedate = c.duedate AND
a.assembliesbatchid > c.assembliesbatchid)
GROUP BY a.assembliesbatchid, a.duedate, a.assemblyid, a.assemblyname,
a.ownerid, a.partid, a.quantity, a.stock, a.units, a.qtyperunit, a.leadfree;

CREATE OR REPLACE VIEW assembliesstockbatchpriorexpected AS
SELECT a.assembliesbatchid, a.duedate, a.assemblyid, a.assemblyname,
a.ownerid, a.partid, a.units, a.quantity, a.stock, a.prioruse,
a.priorusebatch, COALESCE(sum(
CASE
WHEN COALESCE(e.promisedby::timestamp without time
zone::timestamp with time zone, e.requestedby::timestamp without time
zone::timestamp with time zone,
CASE
WHEN e.deliverywks IS NULL THEN f.issuedate
ELSE NULL::date
END::timestamp without time zone::timestamp with time zone,
CASE
WHEN e.deliverywks <> -1 THEN (f.issuedate +
e.deliverywks * 7)::timestamp without time zone::timestamp with time zone
ELSE a.duedate + '1 day'::interval
END) <= a.duedate THEN COALESCE(e.quantity, 0) -
COALESCE(e.deliveredsum, 0)
ELSE NULL::integer
END), 0::bigint) AS expectedbefore, a.qtyperunit
FROM assembliesstockbatchprioruse a
LEFT JOIN (pos f
JOIN poparts e ON f.poid = e.poid AND f.postatusid >= 20 AND
f.postatusid <= 59 AND f.isrfq = false
JOIN manufacturerpartpn g ON g.pnid = e.pnid) ON e.partid = a.partid
GROUP BY a.assembliesbatchid, a.duedate, a.assemblyid, a.assemblyname,
a.ownerid, a.partid, a.quantity, a.stock, a.prioruse, a.units,
a.qtyperunit, a.priorusebatch;

CREATE OR REPLACE VIEW assemblycanbuild AS
SELECT assembliesstockbatchpriorexpected.assembliesbatchid,
CASE
WHEN min(
CASE
WHEN (assembliesstockbatchpriorexpected.stock::double
precision - assembliesstockbatchpriorexpected.prioruse -
assembliesstockbatchpriorexpected.quantity::double precision) >=
0::double precision THEN 100000000::double precision
WHEN
COALESCE(assembliesstockbatchpriorexpected.qtyperunit, 0::double
precision) = 0::double precision OR
(assembliesstockbatchpriorexpected.stock::double precision -
assembliesstockbatchpriorexpected.prioruse) < 0::double precision THEN
0::double precision
ELSE
trunc((assembliesstockbatchpriorexpected.stock::double precision -
assembliesstockbatchpriorexpected.prioruse) /
assembliesstockbatchpriorexpected.qtyperunit)
END) = 100000000::double precision THEN 'All'::character varying
ELSE min(
CASE
WHEN
COALESCE(assembliesstockbatchpriorexpected.qtyperunit, 0::double
precision) = 0::double precision OR
(assembliesstockbatchpriorexpected.stock::double precision -
assembliesstockbatchpriorexpected.prioruse) < 0::double precision THEN
0::double precision
ELSE
trunc((assembliesstockbatchpriorexpected.stock::double precision -
assembliesstockbatchpriorexpected.prioruse) /
assembliesstockbatchpriorexpected.qtyperunit)
END)::character varying
END AS canbuild
FROM assembliesstockbatchpriorexpected
WHERE assembliesstockbatchpriorexpected.quantity <> 0
GROUP BY assembliesstockbatchpriorexpected.assembliesbatchid;

#6Grzegorz Jaśkiewicz
gryzman@gmail.com
In reply to: Sim Zacks (#5)
Re: after vacuum analyze, explain still wrong

On Tue, Jun 23, 2009 at 10:54 AM, Sim Zacks<sim@compulab.co.il> wrote:

Sim Zacks wrote:

Grzegorz Jaśkiewicz wrote:

give us postgresql version as well, maybe show query and at least table layout.

The queries look like (The one I'm trying to run is the last one, which
is based on all the previous ones).:

oh dear, this is big.
I know planner isn't good in propagating rowcount estimations up the
plan. So maybe that's it. But I'll wait for hackers-folk to reply.

--
GJ

#7Bruce Momjian
bruce@momjian.us
In reply to: Grzegorz Jaśkiewicz (#6)
Re: after vacuum analyze, explain still wrong

Actually most of the estimates seem pretty good. There are some that
are a ways off, but the real nasties seem to be these. I'm a bit
confused because it looks like two of your joins don't have Join
Filters -- and one of those is a left join for which I thought that
was impossible.

Are you sure this query is doing what you expect? You have
add_missing_from enabled which will happily bring in additional joins
if you reference a table which isn't already included in the join and
do a full cartesian-product join.

-> Nested Loop
Left Join (cost=145.04..2381.33 rows=1 width=74) (actual
time=38.955..9678.057 rows=3754 loops=1)
Join
Filter: ((c.partid = e.partid) AND leadcompcheck_ab(a.leadfree,
c.leadstateid))
-> Nested
Loop (cost=145.04..2366.26 rows=1 width=62) (actual time=0.791..83.482
rows=3754 loops=1)
->
Nested Loop Left Join (cost=145.04..2365.59 rows=1 width=28) (actual
time=0.721..52.840 rows=3754 loops=1)

Filter: (f.commited IS NOT TRUE)

--
greg
http://mit.edu/~gsstark/resume.pdf

#8Sim Zacks
sim@compulab.co.il
In reply to: Bruce Momjian (#7)
Re: after vacuum analyze, explain still wrong

Greg Stark wrote:

Actually most of the estimates seem pretty good. There are some that
are a ways off, but the real nasties seem to be these. I'm a bit
confused because it looks like two of your joins don't have Join
Filters -- and one of those is a left join for which I thought that
was impossible.

The top half of the explain shows a lot of estimated rows=1 and actual
rows=a lot more, which is why I suspected the stats.
I left join without the filter looks like it is from:

FROM assemblies a
JOIN assembliesbatch d ON d.assemblyid = a.assemblyid
JOIN allocatedassemblies e ON e.assembliesbatchid = d.assembliesbatchid
--HERE
LEFT JOIN partsassembly b ON b.assemblyid = a.assemblyid AND e.partid =
b.partid
--HERE
LEFT JOIN stockperowner_lead_ab() c(partid, ownerid, stock, leadstateid)
ON c.partid = e.partid AND c.ownerid = 1 AND
leadcompcheck_ab(a.leadfree, c.leadstateid)
LEFT JOIN stocklog f ON f.refid = d.batchid AND f.transtypeid = 3 AND
f.partid = e.partid
WHERE (d.assembliesbatchstatusid = ANY (ARRAY[1, 2, 4, 7])) AND
f.commited IS NOT TRUE
GROUP BY d.assembliesbatchid, d.duedate, a.assemblyid, a.assemblyname,
c.ownerid, e.partid, COALESCE(c.stock, 0::bigint), d.units, e.quantity,
a.leadfree;

it looks like it is using an index scan instead of a filter:
-> Index Scan using idx_u_assidpartid on partsassembly b
(cost=0.00..0.28 rows=1 width=16) (actual time=0.011..0.012 rows=1
loops=3705)
Index Cond: ((e.partid = b.partid) AND (b.assemblyid = a.assemblyid))

Are you sure this query is doing what you expect? You have
add_missing_from enabled which will happily bring in additional joins
if you reference a table which isn't already included in the join and
do a full cartesian-product join.

The results of the query are exactly what I expect them to be. I have
manually verified this on multiple occasions with users who wanted
verification that the numbers were correct.
I went through it again to verify and there are no add_missing_from
examples in here. I have that on intentionally, because I use it in
update and delete statements.