Skip to content

PostgreSQL#1195

Merged
Alkarex merged 27 commits intoFreshRSS:devfrom
Alkarex:PostgreSQL
Oct 1, 2016
Merged

PostgreSQL#1195
Alkarex merged 27 commits intoFreshRSS:devfrom
Alkarex:PostgreSQL

Conversation

@Alkarex
Copy link
Copy Markdown
Member

@Alkarex Alkarex commented Aug 2, 2016

#416

#833
Based on @Damstre work #1071
Work in progress. Contributions welcome.

The differences from the original pull request are mainly:

  • Almost no change to MySQL and SQLite (possible optimisation later)
  • Mini compatibility layer in ModelPdo.php
  • More inheritance (so less code specific for PostgreSQL)

@Alkarex Alkarex added the Work in progress 🚧 Wait before merging label Aug 2, 2016
@Alkarex Alkarex added this to the 1.5.2-beta milestone Aug 2, 2016
This was referenced Aug 2, 2016
@Trim
Copy link
Copy Markdown
Contributor

Trim commented Aug 31, 2016

Hello,

I'm very interested in this feature.

Is it still a work in progress ? Could I help you ?
Do you need I install it on my server to do tests on another computer (with Debian Jessie) ?

Thanks,
Adrien

@Alkarex
Copy link
Copy Markdown
Member Author

Alkarex commented Aug 31, 2016

Yes, support for PostgreSQL is planned for the next release https://github.com/FreshRSS/FreshRSS/milestone/19 and testers and/or contributors are needed. Your help would be very much appreciated :-)
The current pull request should already work a bit, although not stable yet, and with some known bugs. You are welcome to give it a try.

Plans:

  • It would be good to try to make the existing SQL code more uniform when possible (between MySQL, SQLite, and PostgreSQL), although not if we loose performances (several queries were manually optimised for performance in MySQL - Tested with 100k+ articles on a Raspberry Pi, and 300k articles on an Atom server).
  • And I would also like to see if we can improve the support for PostgreSQL, regarding performances, especially for text compression, and text index / search (the current version with MySQL is purposely not using native compression and text indexing for supporting older MySQL versions, but this will change in the future).

@Alkarex
Copy link
Copy Markdown
Member Author

Alkarex commented Aug 31, 2016

P.S. @Trim You can check out https://github.com/Alkarex/FreshRSS/tree/PostgreSQL which is the branch used for the current pull request

@anayrat
Copy link
Copy Markdown

anayrat commented Aug 31, 2016

Some remarks in "app/SQL/install.sql.pgsql.php"

  • You should avoid upper case in column name

  • What is the goal of ?

    CREATE OR REPLACE RULE check_constraints_on_entry AS ON INSERT TO "freshrss_admin_entry" WHERE EXISTS(SELECT 1 FROM "freshrss_admin_entry" WHERE guid=NEW.guid) DO INSTEAD NOTHING;

    You should use ON CONFLICT DO NOTHING as rules are expensive.

@Alkarex
Copy link
Copy Markdown
Member Author

Alkarex commented Aug 31, 2016

Thanks for the feedback @anayrat

  • For the column names, I would like to keep as must compatibility as possible with the existing code (MySQL and SQLite do not have problems with lower/upper-case, so some columns ended up with upper-case letters). It should work as it is in the current pull request, though.
  • A good part of the PostgreSQL-specific code comes from 416 postgresql support #1071 . I have not tried to optimise it yet. Input welcome.

@Trim
Copy link
Copy Markdown
Contributor

Trim commented Sep 1, 2016

Ok, so I've just installed it.

During the install process, I've seen in the Checks view with this information:

You have PDO and at least one of the supported drivers (pdo_mysql, pdo_sqlite).

I think, it should have something like pdo_pgsql too.

In the Database Configuration view, I can only choose between MySQL and PostgreSQL.
Should I have been able to choose SQLite too ?

Then, I was able to import my 68 feeds from my production instance and it was able to reach 1804 articles without any issue apparently (apache2 error logs are empty).

I've check apache2 access log and I've found the import took this time:

fe80::aaaa - - [01/Sep/2016:15:43:27 +0200] "POST /i/?c=importExport&a=import HTTP/1.1" 302 586 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
→ many POST with a=actualize between these lines
fe80::aaaa - - [01/Sep/2016:15:46:51 +0200] "GET /i/ HTTP/1.1" 200 48354 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"

So it took around 3.5 minutes which seems not bad :)
(I have no idea the time needed to do so with the MySQL installation as it has already all the articles...)

Thanks !

@Alkarex
Copy link
Copy Markdown
Member Author

Alkarex commented Sep 3, 2016

@Trim If you could not select SQLite, it is probably that you did not install pdo_sqlite.
I have just updated a bunch of messages related to PostgreSQL 2fa5ef3

@Alkarex Alkarex merged commit 0ccf100 into FreshRSS:dev Oct 1, 2016
@Alkarex Alkarex deleted the PostgreSQL branch October 1, 2016 17:35
@Alkarex
Copy link
Copy Markdown
Member Author

Alkarex commented Oct 1, 2016

@Trim @anayrat @Damstre : PostgreSQL has landed in the dev branch of FreshRSS :-) Tests welcome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants