PostgreSQL Database Admin: MVCC & Indexing

๐Ÿ’ก PostgreSQL Database Admin: MVCC & Indexing โ€” Executive Summary

Technical architectural analysis, practical configuration manifests, and deployment instructions for PostgreSQL Database Admin: MVCC & Indexing.

Basic
sql
CREATE USER keycloak PASSWORD 'keycloak' SUPERUSER;
CREATE DATABASE test_keycloak OWNER keycloak;

-- **DB ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ์ƒ์„ฑ**
CREATE DATABASE *dbname* OWNER *ownnerName*
DB Access
bash
-- **DB ์ ‘์†**
psql -U [DB์‚ฌ์šฉ์ž๊ณ„์ •] [๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค๋ช…]

-- **Postgre SQL shell ์ง„์ž…์‹œ**
psql [์Šคํ‚ค๋งˆ๋ช…]
bash
psql #default account postgres
psql -U {user} -d {database} -c "{query}" #postgres ์œ ์ €, airflow Database(schema)
DB System
Grant
sql
GRANT ALL PRIVILEGES ON ALL TABLES IN schema public TO postgres;
GRANT ALL PRIVILEGES ON DATABASE keycloak TO postgres;
Show
sql
show stuffs

-- Show CONFIG_PATH
show config_file;
Table
sql
-- **DB ์‹คํ–‰์ค‘์ธ ์ฟผ๋ฆฌ ์กฐํšŒ**
SELECT * FROM pg_stat_activity ORDER BY query_start ASC;

-- Setting ์กฐํšŒ
SELECT name, setting FROM pg_settings;

--
SELECT * FROM PG_DATABASE;
SELECT * FROM PG_USER;
select * from PG_ROLES;

-- ํ…Œ์ด๋ธ” ๋ฆด๋ ˆ์ด์…˜ ์‚ฌ์ด์ฆˆ ์กฐํšŒ
SELECT   schemaname, tablename, pg_size_pretty(pg_total_relation_size( schemaname || '.' || tablename )) AS size
FROM     pg_tables
WHERE    NOT (schemaname = 'pg_catalog' OR schemaname = 'information_schema')
ORDER BY pg_total_relation_size( schemaname || '.' || tablename ) DESC;
Expression
sql
-- **DB ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ์ถœ๋ ฅ**
\l or \list

-- **DB ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ์„ ํƒ**
\c [๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ๋ช…]

-- **DB ํ…Œ์ด๋ธ” ์ถœ๋ ฅ**
\dt

-- **ํ…Œ์ด๋ธ” ๊ตฌ์กฐ ์กฐํšŒ**: ์˜ค๋ผํด์˜ DESCRIBE TABLE # \d+ ํ…Œ์ด๋ธ”๋ช… # \d+ ํ…Œ์ด๋ธ”๋ช…
-- **DB ํ•ด๋‹น ํ…Œ์ด๋ธ” ์ •๋ณด ์ถœ๋ ฅ**
\d [ํ…Œ์ด๋ธ”๋ช…]
\d+ [ํ…Œ์ด๋ธ”๋ช…]

-- **DB ๋‚˜๊ฐ€๊ธฐ(์ข…๋ฃŒ)**
\q

-- **DB ์‚ฌ์šฉ์ž ๊ถŒํ•œ ์ •๋ณด**
\du

-- **DB ์ถœ๋ ฅ ๋ณ€๊ฒฝ**
\x

-- **DB ์ฟผ๋ฆฌ ๊ฒฐ๊ณผ๊ฐ’ ํŒŒ์ผ ์ €์žฅ(์ฟผ๋ฆฌ ๊ฒฐ๊ณผ๊ฐ’์„ ์ถœ๋ ฅํ•˜์ง€ ์•Š๊ณ  ํŒŒ์ผ์— ์ €์žฅ๋จ)**
\o [ํŒŒ์ผ๊ฒฝ๋กœ]
sql
\l # list of databases(schema)
\r # clear query buffer
\q # quit session
\{}+ # additional information
\dt # List of relations(type=table)
\ds # List of relations(type=sequence)
\du # List of roles on Users
\d {table} # Show scheme for Table public.job

Backup & Restore
bash
# **SQL๋ฌธ ํŒŒ์ผ์„ ์‹คํ–‰** $ psql -U [์‚ฌ์šฉ์ž๋ช…] [DB๋ช…] < [SQLํŒŒ์ผ๋ช…]
# **ํŒŒ์ผ๋กœ๋ถ€ํ„ฐ DB๋กœ dump, ๋˜๋Š” DB๋กœ๋ถ€ํ„ฐ ํŒŒ์ผ๋กœ dump** # COPY ํ…Œ์ด๋ธ”๋ช… [(์ปฌ๋Ÿผ๋ช…1, ์ปฌ๋Ÿผ๋ช…2,...)] FROM 'ํŒŒ์ผ๋ช…' # COPY ํ…Œ์ด๋ธ”๋ช… [(์ปฌ๋Ÿผ๋ช…1, ์ปฌ๋Ÿผ๋ช…2,...)] TO 'ํŒŒ์ผ๋ช…' ๊ตฌ๋ถ„์ž๋ฅผ ์ง€์ •ํ•˜์ง€ ์•Š์œผ๋ฉด ๊ธฐ๋ณธ์ ์œผ๋กœ ํƒญ์œผ๋กœ ์ง€์ •๋จ. ์ „์ฒด ์˜ต์…˜์€ ์•„๋ž˜์™€ ๊ฐ™์Œ.[http://www.postgresql.org/docs/9.2/static/sql-copy.html](http://www.postgresql.org/docs/9.2/static/sql-copy.html)์ฐธ๊ณ 
# **ํŒŒ์ผ๋กœ dump**
pg_dump -d [db๋ช…]  -U [user] -F t > [ํŒŒ์ผ๋ช….tar]

- d, --dbname : Backupํ•  Database ๋ช….
- h, --host : Database ์ฃผ์†Œ.
- U, --username : Database ์ ‘์† ์‹œ User ID
- F, --format : Backup Format. ํ•„์ž๋Š” ์ฃผ๋กœ tar ํŒŒ์ผ๋กœ backupํ•˜๊ธฐ ๋•Œ๋ฌธ์— 't'๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค.
- f, --file : Backup File Name
- t, --table : ํŠน์ • Table๋งŒ Backupํ•˜๋ คํ•  ๋•Œ ๋Œ€์ƒ์ด ๋˜๋Š” Table ๋ช…
- j, --jobs : Backup ์‹œ ๋ณ‘๋ ฌ ์ฒ˜๋ฆฌ ์—ฌ๋ถ€์™€ ๊ทธ ์ •๋„.
- v, --verbose : ์ง„ํ–‰ ๊ณผ์ • ํ‘œ์‹œ.
- **dump to restore**

pg_restore -h localhost -U postgres -C -d postgres -F t [dumpFile]

- d, --dbname : Restoreํ•˜๋Š” Database ๋ช….
- h, --host : Database ์ฃผ์†Œ.
- U, --username : Database ์ ‘์† ์‹œ User ID
- F, --format : Restore File์˜ Format.
- t, --table : ํŠน์ • Table๋งŒ Restoreํ•˜๋ คํ•  ๋•Œ ๋Œ€์ƒ์ด ๋˜๋Š” Table ๋ช…
- j, --jobs : Restore ์‹œ ๋ณ‘๋ ฌ ์ฒ˜๋ฆฌ ์—ฌ๋ถ€์™€ ๊ทธ ์ •๋„.
- v, --verbose : ์ง„ํ–‰ ๊ณผ์ • ํ‘œ์‹œ
- C, --create : Target DB๋ฅผ ์ƒˆ๋กœ ๋งŒ๋“ค๋ฉด์„œ Restoration ์ง„ํ–‰.
- c, --clean : Restoration ์‹œ์— ๊ฐ™์€ ์ด๋ฆ„์˜ Database Object๊ฐ€ ๋ฐœ๊ฒฌ๋˜๋ฉด Drop ํ›„์— Createํ•˜๊ฒŒ ํ•จ.
- O, --no-owner : ์›๋ณธ DB์˜ Owner๊ฐ€ ๋ณต๊ตฌํ•  ์œ„์น˜์— ์กด์žฌํ•˜์ง€ ์•Š์„ ๊ฒฝ์šฐ ๋ณต๊ตฌ ์‹œ ๋‹ค๋Ÿ‰์˜ ์—๋Ÿฌ๊ฐ€ ๋ฐœ์ƒํ•œ๋‹ค. ์ด๋ฅผ ๋ง‰๊ธฐ ์œ„ํ•ด DB ๋ณต๊ตฌ์‹œ OWNER๋ฅผ ๋ช…์‹œํ•˜์ง€ ์•Š๊ณ  ์ง„ํ–‰ํ•˜๊ฒŒ ํ•จ.
Spacing

โ€” Clear storage used by DB after Delete

References and Guidelines (NOTE)

In PostgreSQL, both โ€˜VACUUMโ€™ and โ€˜VACUUM FULLโ€™ are used to manage and optimize database storage, but their behavior and purpose are different. The differences between the two are as follows:

  1. VACUUM:

    โ€˜VACUUMโ€™ is a command used to secure storage space and improve performance by removing dead rows, updating statistics, etc. When you perform a data modification (such as DELETE, UPDATE, or INSERT), PostgreSQL marks the previous version of the row as โ€œdeadโ€ but does not immediately remove it. This can lead to fragmentation and inefficiency.

The โ€˜VACUUMโ€™ command has several modes and can be run without options to perform routine cleanup, remove dead rows, and update statistics for all tables and indexes in the database. This is a non-blocking operation. This means it can be done while the database is online and accessible.

sql
VACUUM;

You can also target individual tables or indexes using specific options with โ€˜VACUUMโ€™.

  • โ€˜VACUUM FULLโ€™ typically locks the table only for the duration of the operation and rewrites the entire table, which can be time-consuming and resource-intensive, so it is generally not recommended for general use. This is typically used when significant amounts of space need to be reclaimed after major data modifications.
    2. VACUUM FULL:

    VACUUM FULL is a more aggressive form of the VACUUM command. Reclaim storage space by creating a new copy of the table, copying only the active rows, and then replacing the original table with the new copy. This process helps reduce fragmentation and optimize storage. However, because you are creating a new table, you will need an exclusive lock on the table being VACUUMed to effectively prevent any other operations on the table while the VACUUM FULL process is in progress.

sql
VACUUM FULL your_table;

Due to exclusive locking and potential performance impact, โ€˜VACUUM FULLโ€™ is generally used with caution and only in situations where significant storage reclamation is required.

In summary, consider using VACUUM on a regular basis for routine maintenance and space reclamation, and using VACUUM FULL sparingly, only when you need to perform a large cleanup with the intention of locking the table exclusively for a period of time. Always perform these operations in a controlled manner, considering the impact on the database and ongoing operations.

Features
Airflow
  • delete failed tasks
bash
# psql node
  sudo -i
  su -- postgres
  psql -U postgres -d airflow -c "delete from dag_run where dag_id='${DAG_ID}' AND state='failed';"

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *