[GSoC]The project summary
Hi mentors and hackers,
The final review is coming. Here is the project summary for the thrift
plugin work for Postgres database. Please let me know if there are anything
missing for the final review.
1. Implement the thrift binary protocol for both simple data structures
(e.g., int, double) and complex data structures (e.g., list, map and
struct) in pg_thrift plugin. The interface is byte based which means user
need to pass in a byte and can use rich apis to parse out required fields.
2. Implement the thrift compact protocol for both simple data structures
and complex data structures. The interface is also byte based and user can
use rich apis to parse out fields.
3. A set of APIs for both binary protocol and compact protocol to parse out
fields with kinds of types.
4. A customized thrift type (thrift_binary) where user specifies json, but
stores in the format of byte. This type makes the plugin more user
friendly, currently we support simple getter on top of this type. There are
some improvements that can be done in the future to make the type support
more operations.
5. Set up CI to continuously compile for each commit. Currently the plugin
works in 9.4, 10, and 11.
6. A set of unit tests to cover most important use cases(
https://github.com/charles-cui/pg_thrift/blob/master/sql/pg_thrift.sql).
7. A detailed document to showcase how to use this plugin (
https://github.com/charles-cui/pg_thrift/blob/master/README.md).
From this document, user knows how to install pg_thrift, how to parse out
required fields from byte using provided api, how to build index based on
the thrift bytes by the use of the api, and how to use the customized
thrift type.
Hello Charles,
Thanks for keeping us informed. As you probably already discovered the
email I used previously doesn't work any longer. Please add
afiskon@gmail.com to CC instead.
I will take a look tomorrow (it's pretty late in my timezone currently).
On Sun, Aug 5, 2018 at 9:05 PM, Charles Cui <charles.cui1984@gmail.com>
wrote:
Hi mentors and hackers,
The final review is coming. Here is the project summary for the thrift
plugin work for Postgres database. Please let me know if there are anything
missing for the final review.
1. Implement the thrift binary protocol for both simple data structures
(e.g., int, double) and complex data structures (e.g., list, map and
struct) in pg_thrift plugin. The interface is byte based which means user
need to pass in a byte and can use rich apis to parse out required fields.
2. Implement the thrift compact protocol for both simple data structures
and complex data structures. The interface is also byte based and user can
use rich apis to parse out fields.
3. A set of APIs for both binary protocol and compact protocol to parse
out fields with kinds of types.
4. A customized thrift type (thrift_binary) where user specifies json, but
stores in the format of byte. This type makes the plugin more user
friendly, currently we support simple getter on top of this type. There are
some improvements that can be done in the future to make the type support
more operations.
5. Set up CI to continuously compile for each commit. Currently the plugin
works in 9.4, 10, and 11.
6. A set of unit tests to cover most important use cases(
https://github.com/charles-cui/pg_thrift/blob/master/sql/pg_thrift.sql).
7. A detailed document to showcase how to use this plugin (
https://github.com/charles-cui/pg_thrift/blob/master/README.md).
From this document, user knows how to install pg_thrift, how to parse out
required fields from byte using provided api, how to build index based on
the thrift bytes by the use of the api, and how to use the customized
thrift type.
--
Best regards,
Aleksander Alekseev
Hello Charles,
Personally I believe the project is in a pretty good shape, well done!
However I would recommend to replace all sprintf calls with it's safer
version, snprintf. Also please consider to run the code through Clang
Static Analyzer and Valgrind one more time just to make sure there are no
stupid mistakes that will blow up someone's PostgreSQL instance :)
On Sun, Aug 5, 2018 at 11:04 PM, Aleksander Alekseev <afiskon@gmail.com>
wrote:
Hello Charles,
Thanks for keeping us informed. As you probably already discovered the
email I used previously doesn't work any longer. Please add
afiskon@gmail.com to CC instead.I will take a look tomorrow (it's pretty late in my timezone currently).
On Sun, Aug 5, 2018 at 9:05 PM, Charles Cui <charles.cui1984@gmail.com>
wrote:Hi mentors and hackers,
The final review is coming. Here is the project summary for the thrift
plugin work for Postgres database. Please let me know if there are anything
missing for the final review.
1. Implement the thrift binary protocol for both simple data structures
(e.g., int, double) and complex data structures (e.g., list, map and
struct) in pg_thrift plugin. The interface is byte based which means user
need to pass in a byte and can use rich apis to parse out required fields.
2. Implement the thrift compact protocol for both simple data structures
and complex data structures. The interface is also byte based and user can
use rich apis to parse out fields.
3. A set of APIs for both binary protocol and compact protocol to parse
out fields with kinds of types.
4. A customized thrift type (thrift_binary) where user specifies json,
but stores in the format of byte. This type makes the plugin more user
friendly, currently we support simple getter on top of this type. There are
some improvements that can be done in the future to make the type support
more operations.
5. Set up CI to continuously compile for each commit. Currently the
plugin works in 9.4, 10, and 11.
6. A set of unit tests to cover most important use cases(
https://github.com/charles-cui/pg_thrift/blob/master/sql/pg_thrift.sql).
7. A detailed document to showcase how to use this plugin (
https://github.com/charles-cui/pg_thrift/blob/master/README.md).
From this document, user knows how to install pg_thrift, how to parse out
required fields from byte using provided api, how to build index based on
the thrift bytes by the use of the api, and how to use the customized
thrift type.--
Best regards,
Aleksander Alekseev
--
Best regards,
Aleksander Alekseev
got it, will do that asap. Thanks for the comments!
On Mon, Aug 6, 2018, 7:06 AM Aleksander Alekseev <afiskon@gmail.com> wrote:
Show quoted text
Hello Charles,
Personally I believe the project is in a pretty good shape, well done!
However I would recommend to replace all sprintf calls with it's safer
version, snprintf. Also please consider to run the code through Clang
Static Analyzer and Valgrind one more time just to make sure there are no
stupid mistakes that will blow up someone's PostgreSQL instance :)On Sun, Aug 5, 2018 at 11:04 PM, Aleksander Alekseev <afiskon@gmail.com>
wrote:Hello Charles,
Thanks for keeping us informed. As you probably already discovered the
email I used previously doesn't work any longer. Please add
afiskon@gmail.com to CC instead.I will take a look tomorrow (it's pretty late in my timezone currently).
On Sun, Aug 5, 2018 at 9:05 PM, Charles Cui <charles.cui1984@gmail.com>
wrote:Hi mentors and hackers,
The final review is coming. Here is the project summary for the
thrift plugin work for Postgres database. Please let me know if there are
anything missing for the final review.
1. Implement the thrift binary protocol for both simple data structures
(e.g., int, double) and complex data structures (e.g., list, map and
struct) in pg_thrift plugin. The interface is byte based which means user
need to pass in a byte and can use rich apis to parse out required fields.
2. Implement the thrift compact protocol for both simple data structures
and complex data structures. The interface is also byte based and user can
use rich apis to parse out fields.
3. A set of APIs for both binary protocol and compact protocol to parse
out fields with kinds of types.
4. A customized thrift type (thrift_binary) where user specifies json,
but stores in the format of byte. This type makes the plugin more user
friendly, currently we support simple getter on top of this type. There are
some improvements that can be done in the future to make the type support
more operations.
5. Set up CI to continuously compile for each commit. Currently the
plugin works in 9.4, 10, and 11.
6. A set of unit tests to cover most important use cases(
https://github.com/charles-cui/pg_thrift/blob/master/sql/pg_thrift.sql
).
7. A detailed document to showcase how to use this plugin (
https://github.com/charles-cui/pg_thrift/blob/master/README.md).
From this document, user knows how to install pg_thrift, how to parse
out required fields from byte using provided api, how to build index based
on the thrift bytes by the use of the api, and how to use the customized
thrift type.--
Best regards,
Aleksander Alekseev--
Best regards,
Aleksander Alekseev
commit to change sprintf to snprintf
https://github.com/charles-cui/pg_thrift/commit/1de265a661ab3c61aa593d7e99d3a313024170fc
2018-08-06 8:03 GMT-07:00 Charles Cui <charles.cui1984@gmail.com>:
Show quoted text
got it, will do that asap. Thanks for the comments!
On Mon, Aug 6, 2018, 7:06 AM Aleksander Alekseev <afiskon@gmail.com>
wrote:Hello Charles,
Personally I believe the project is in a pretty good shape, well done!
However I would recommend to replace all sprintf calls with it's safer
version, snprintf. Also please consider to run the code through Clang
Static Analyzer and Valgrind one more time just to make sure there are no
stupid mistakes that will blow up someone's PostgreSQL instance :)On Sun, Aug 5, 2018 at 11:04 PM, Aleksander Alekseev <afiskon@gmail.com>
wrote:Hello Charles,
Thanks for keeping us informed. As you probably already discovered the
email I used previously doesn't work any longer. Please add
afiskon@gmail.com to CC instead.I will take a look tomorrow (it's pretty late in my timezone currently).
On Sun, Aug 5, 2018 at 9:05 PM, Charles Cui <charles.cui1984@gmail.com>
wrote:Hi mentors and hackers,
The final review is coming. Here is the project summary for the
thrift plugin work for Postgres database. Please let me know if there are
anything missing for the final review.
1. Implement the thrift binary protocol for both simple data structures
(e.g., int, double) and complex data structures (e.g., list, map and
struct) in pg_thrift plugin. The interface is byte based which means user
need to pass in a byte and can use rich apis to parse out required fields.
2. Implement the thrift compact protocol for both simple data
structures and complex data structures. The interface is also byte based
and user can use rich apis to parse out fields.
3. A set of APIs for both binary protocol and compact protocol to parse
out fields with kinds of types.
4. A customized thrift type (thrift_binary) where user specifies json,
but stores in the format of byte. This type makes the plugin more user
friendly, currently we support simple getter on top of this type. There are
some improvements that can be done in the future to make the type support
more operations.
5. Set up CI to continuously compile for each commit. Currently the
plugin works in 9.4, 10, and 11.
6. A set of unit tests to cover most important use cases(
https://github.com/charles-cui/pg_thrift/blob/master/sql/pg_thrift.sql
).
7. A detailed document to showcase how to use this plugin (
https://github.com/charles-cui/pg_thrift/blob/master/README.md).
From this document, user knows how to install pg_thrift, how to parse
out required fields from byte using provided api, how to build index based
on the thrift bytes by the use of the api, and how to use the customized
thrift type.--
Best regards,
Aleksander Alekseev--
Best regards,
Aleksander Alekseev
clang static analysis looks good
~/Desktop/checker-279/bin/scan-build make && make install && make
installcheck
scan-build: Using '/Users/wyx/Desktop/checker-279/bin/clang' for static
analysis
/Users/wyx/Desktop/checker-279/bin/../libexec/ccc-analyzer -Wall
-Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement
-Wendif-labels -Wmissing-format-attribute -Wformat-security
-fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -g -O2
-Wall -std=c99 -I. -I./
-I/usr/local/Cellar/postgresql@11/11beta2/include/server
-I/usr/local/Cellar/postgresql@11/11beta2/include/internal
-I/usr/local/Cellar/icu4c/62.1/include -I/usr/local/opt/gettext/include
-I/usr/local/opt/icu4c/include -I/usr/local/opt/openldap/include
-I/usr/local/opt/openssl/include -I/usr/local/opt/readline/include
-I/usr/local/opt/tcl-tk/include -c -o pg_thrift.o pg_thrift.c
/Users/wyx/Desktop/checker-279/bin/../libexec/ccc-analyzer -Wall
-Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement
-Wendif-labels -Wmissing-format-attribute -Wformat-security
-fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -bundle
-multiply_defined suppress -o pg_thrift.so pg_thrift.o
-L/usr/local/Cellar/postgresql@11/11beta2/lib -L/usr/local/opt/gettext/lib
-L/usr/local/opt/icu4c/lib -L/usr/local/opt/openldap/lib
-L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib
-L/usr/local/opt/tcl-tk/lib -Wl,-dead_strip_dylibs -bundle_loader
/usr/local/Cellar/postgresql@11/11beta2/bin/postgres
scan-build: Removing directory
'/var/folders/z5/6q03lw915_v6b26zwmplw1m80000gn/T/scan-build-2018-08-06-223529-18582-1'
because it contains no reports.
scan-build: No bugs found.
/bin/sh /usr/local/Cellar/postgresql@11/11beta2/lib/pgxs/src/makefiles/../../config/install-sh
-c -d '/usr/local/Cellar/postgresql@11/11beta2/lib'
/bin/sh /usr/local/Cellar/postgresql@11/11beta2/lib/pgxs/src/makefiles/../../config/install-sh
-c -d '/usr/local/Cellar/postgresql@11/11beta2/share/extension'
/bin/sh /usr/local/Cellar/postgresql@11/11beta2/lib/pgxs/src/makefiles/../../config/install-sh
-c -d '/usr/local/Cellar/postgresql@11/11beta2/share/extension'
/usr/bin/install -c -m 755 pg_thrift.so '/usr/local/Cellar/postgresql@11
/11beta2/lib/pg_thrift.so'
/usr/bin/install -c -m 644 .//pg_thrift.control
'/usr/local/Cellar/postgresql@11/11beta2/share/extension/'
/usr/bin/install -c -m 644 .//pg_thrift--1.0.sql
'/usr/local/Cellar/postgresql@11/11beta2/share/extension/'
/usr/local/Cellar/postgresql@11/11beta2/lib/pgxs/src/makefiles/../../src/test/regress/pg_regress
--inputdir=./ --bindir='/usr/local/Cellar/postgresql@11/11beta2/bin'
--dbname=contrib_regression
pg_thrift
(using postmaster on Unix socket, default port)
============== dropping database "contrib_regression" ==============
DROP DATABASE
============== creating database "contrib_regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries ==============
test pg_thrift ... ok
=====================
All 1 tests passed.
=====================
2018-08-06 17:56 GMT-07:00 Charles Cui <charles.cui1984@gmail.com>:
Show quoted text
commit to change sprintf to snprintf
https://github.com/charles-cui/pg_thrift/commit/
1de265a661ab3c61aa593d7e99d3a313024170fc2018-08-06 8:03 GMT-07:00 Charles Cui <charles.cui1984@gmail.com>:
got it, will do that asap. Thanks for the comments!
On Mon, Aug 6, 2018, 7:06 AM Aleksander Alekseev <afiskon@gmail.com>
wrote:Hello Charles,
Personally I believe the project is in a pretty good shape, well done!
However I would recommend to replace all sprintf calls with it's safer
version, snprintf. Also please consider to run the code through Clang
Static Analyzer and Valgrind one more time just to make sure there are no
stupid mistakes that will blow up someone's PostgreSQL instance :)On Sun, Aug 5, 2018 at 11:04 PM, Aleksander Alekseev <afiskon@gmail.com>
wrote:Hello Charles,
Thanks for keeping us informed. As you probably already discovered the
email I used previously doesn't work any longer. Please add
afiskon@gmail.com to CC instead.I will take a look tomorrow (it's pretty late in my timezone currently).
On Sun, Aug 5, 2018 at 9:05 PM, Charles Cui <charles.cui1984@gmail.com>
wrote:Hi mentors and hackers,
The final review is coming. Here is the project summary for the
thrift plugin work for Postgres database. Please let me know if there are
anything missing for the final review.
1. Implement the thrift binary protocol for both simple data
structures (e.g., int, double) and complex data structures (e.g., list, map
and struct) in pg_thrift plugin. The interface is byte based which means
user need to pass in a byte and can use rich apis to parse out required
fields.
2. Implement the thrift compact protocol for both simple data
structures and complex data structures. The interface is also byte based
and user can use rich apis to parse out fields.
3. A set of APIs for both binary protocol and compact protocol to
parse out fields with kinds of types.
4. A customized thrift type (thrift_binary) where user specifies json,
but stores in the format of byte. This type makes the plugin more user
friendly, currently we support simple getter on top of this type. There are
some improvements that can be done in the future to make the type support
more operations.
5. Set up CI to continuously compile for each commit. Currently the
plugin works in 9.4, 10, and 11.
6. A set of unit tests to cover most important use cases(
https://github.com/charles-cui/pg_thrift/blob/master/sql/pg_thrift.sql
).
7. A detailed document to showcase how to use this plugin (
https://github.com/charles-cui/pg_thrift/blob/master/README.md).
From this document, user knows how to install pg_thrift, how to parse
out required fields from byte using provided api, how to build index based
on the thrift bytes by the use of the api, and how to use the customized
thrift type.--
Best regards,
Aleksander Alekseev--
Best regards,
Aleksander Alekseev