Skip to content

Implement import of databases in CSV (Comma Separated Values) format#146

Merged
phoerious merged 10 commits intokeepassxreboot:developfrom
seatedscribe:feature/import-csv-format
Mar 16, 2017
Merged

Implement import of databases in CSV (Comma Separated Values) format#146
phoerious merged 10 commits intokeepassxreboot:developfrom
seatedscribe:feature/import-csv-format

Conversation

@seatedscribe
Copy link
Copy Markdown
Contributor

@seatedscribe seatedscribe commented Jan 9, 2017

Description

IMPORT CSV FORMAT

It is now possible for KeepassXC to import databases from CSV text files.

PARSER

I developed the parser with a BNF approach for syntax validation; just for reference, this is the RFC grammar as provided by the IETF (RFC 4180):

file = [header CRLF] record *(CRLF record) [EOF] #[CRLF]
header = name *(COMMA name)
record = field *(COMMA field)
name = field
field = (escaped | non-escaped)
escaped = DQUOTE *(TEXTDATA | COMMA | CR | LF | 2DQUOTE) DQUOTE
non-escaped = *TEXTDATA
COMMA = %x2C
CR = %x0D
DQUOTE = %x22
LF = %x0A
CRLF = CR LF
TEXTDATA = %x20-21 | %x23-2B | %x2D-7E

I needed to validate the two most used versions - the "doubling quotes" and
the "escape character" - into the same engine, so I slightly modified
the grammar to better reflect actual code implementation. I also extended it
to validate the case of "escaped characters". This is the result:

file = record *(CRLF record) EOF
record = field *(separator field) | comment --e.g. ,,A,B has 2 empty fields and is valid
field = simple | quoted
simple = TEXT -- e.g. f""ie"l"""d is OK
quoted = qualifier escaped qualifier
escaped = ESC_TEXT *(escape_mark ESC_TEXT)
escape_mark = (2qualifier | BACKSLASH qualifier)
qualifier = one of {DQUOTE, COLON, BACKSLASH}
separator = one of {COMMA, COLON, SEMICOLON}
comment = *(SPACE | TAB) COMMENT
COMMA = %x2C --i.e. ','
COLON = %x58 --i.e. ':'
SEMICOLON = %x59 --i.e. ';'
COMMENT = %x23 --i.e. '#'
DQUOTE = %x22 --i.e. '"'
BACKSLASH = %x5C --i.e. ''
CR = %x0D
LF = %x0A
SPACE = %x20
CRLF = CR LF | LF
TEXT = *(any char except CRLF, separator, EOF) (also TEXT="" shall be valid)
ESC_TEXT = +(any char except qualifier, EOF)

GUI

I enjoyed putting some effort in GUI design to make the user experience enjoyable and at the same time to be future-proof and avoid reinventing the wheel; for further details look at the comment around CsvImportWidget::m_columnHeader.

TEST

Last but not least, in particular when we talk about security, TDD is a must. I think I put for the parser testing all the conceivable cases together and get to successfully pass them (at the end).

Hope will help and be useful!

Regards,
Enrico

Motivation and Context

This is the original PR made for KeepassX rebased on keepassxc/develop branch.

How Has This Been Tested?

Test cases are covered in the appropriate file under tests/ directory.
Mainly manual user tests for the GUI.

Types of changes

  • ❎ Bug fix (non-breaking change which fixes an issue)
  • ✅ New feature (non-breaking change which adds functionality)
  • ❎ Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • ✅ I have read the CONTRIBUTING document. [REQUIRED]
  • ✅ My code follows the code style of this project. [REQUIRED]
  • ✅ All new and existing tests passed. [REQUIRED]
  • ✅ : My change requires a change to the documentation.
  • ❎ I have updated the documentation accordingly.
  • ✅ I have added tests to cover my changes.

@droidmonkey droidmonkey modified the milestone: v2.2.0 Jan 9, 2017
@dgt79
Copy link
Copy Markdown

dgt79 commented Jan 25, 2017

Hi, I think this is a long overdue feature that is much needed by pretty much anyone migrating to keepassx from another password manager (1password, etc).

@phoerious
Copy link
Copy Markdown
Member

We will merge this pull request into 2.2.0 after we reviewed it in-depth.

m_isFileLoaded = false;
}
else {
device->close();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe indentation is missing here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

skipLine();
return;
}
else {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe no need to enclose in an else block here, because of the early return just above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return is expected only when if (isComment()) is true. I think you made a typo 👀

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe at the end of the if (isComment()), you return from the function. So, no need to enclose the rest of the function in a else block. This is mostly a matter of style though.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@louib Ah, now I get it. Yes, it's a matter of style. My point is that with the else 'label' you stick with the usual code flow and the eye can catch the link/separation of the two logical branches immediately.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Early return guards are nicer and IMHO also easier to read. Unnecessary indented else blocks add more visual complexity that can be avoided.

#include <QTextStream>

typedef QStringList csvrow;
typedef QList<csvrow> csvtable;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe I think these need to be PascalCase, since they define types. e.g. typedef QStringList CsvRow

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

#include "gui/MessageBox.h"

#include <QFile>
#include <QFileInfo>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe I think the general guideline for imports in the project is:

  1. _ui and associated .h file
  2. QT imports
  3. imports from the project.
    These three sections separated by a blank line.

Look at SearchWidget.cpp's import section, for example.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

Copy link
Copy Markdown
Member

@phoerious phoerious Feb 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't really have a guideline for the order of imports. Most of the code uses what louib describes, so maybe it's best to stick with it for now. However, I much dislike it. I would much more prefer

  1. ui and associated .h file
  2. project includes
  3. Qt imports
  4. system imports

Database *m_db;

KeePass2Writer m_writer;
static const QStringList m_columnheader;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe m_columnHeader

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

QFont font = m_ui->labelHeadline->font();
font.setBold(true);
font.setPointSize(font.pointSize() + 2);
m_ui->labelHeadline->setFont(font);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe can this go in the .ui file?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

int x=i%combo_rows;
int y= 2*(i/combo_rows);
m_ui->gridLayout_combos->addWidget(label, x, y);
m_ui->gridLayout_combos->addWidget(combo, x, y+1);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe missing spaces around operators in the last couple of lines.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

m_ui->tableViewFields->resizeRowsToContents();
m_ui->tableViewFields->resizeColumnsToContents();

for (int c=0; c<m_ui->tableViewFields->horizontalHeader()->count(); ++c) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe spaces around operators here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗 I also made the same correction here and there in my files

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you're done with your changes, can you rebase your branch on develop, so we can re-review it, fix any remaining issues and then finally merge this baby?

}
else {
//SHOULD NEVER GET HERE
m_db->rootGroup()->setName("ROOT_FALLBACK");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe but we do get here, if we try importing an empty csv file...

}


void CsvImportWidget::checkGroupNames() {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe since this function sets the root group of the db, why not name it setRootGroup?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

@louib
Copy link
Copy Markdown
Member

louib commented Feb 3, 2017

@seatedscribe thanks for the PR! Some comments:

When trying to load the following file (created by exporting from KeePassXC

"Group","Title","Username","Password","URL","Notes"
"Root","entry1","username1","password1","","Some notes here"
"Root","entry2","username2","password2","",""
"Root","entry3","","password3","https://website.com",""
"Root","entry4","","password4","https://website.com",""

The program seg faults if skip first rows is set to 1 before trying to import the csv file.

About that field, is it really a use case to skip anything else than the first (header) line? Why not just a checkbox Skip header line?

When cancelling the CSV import, the user gets asked 2 times to discard changes instead of one.

@louib
Copy link
Copy Markdown
Member

louib commented Feb 3, 2017

Some screenshots of the new widgets:

screenshot from 2017-02-02 23 03 51

screenshot from 2017-02-02 21 38 49

@TheZ3ro TheZ3ro mentioned this pull request Feb 11, 2017
Copy link
Copy Markdown
Member

@phoerious phoerious left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I marked a few more things that need to be changed. Other than those, I'm really excited to merge. Good work!

bool CsvParser::parseFile() {
parseRecord();
while (!m_isEof)
{
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put opening braces of loops and conditions in the same line as the loop header (issue also found in other places of the code).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

if (isQualifier(m_ch)) {
parseQuoted(field);
}
else {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For new code, please write if/else as

if (condition) {
    // ....
} else {
    // ...
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗 also for old code now

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, as far as KeePassXC is concerned, everything in this PR is new code. 😉

void CsvParser::fillColumns() {
//fill the rows with lesser columns with empty fields

for (int i=0; i<m_table.size(); ++i) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add spaces around operators (also found in many other parts of the code)

Copy link
Copy Markdown
Contributor Author

@seatedscribe seatedscribe Feb 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done already

void setFieldSeparator(const QChar c);
void setTextQualifier(const QChar c);
void setBackslashSyntax(bool set);
int getFileSize() const;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't over-indent

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uff, 🆗 and you dudes, don't over-analyze 😄

bool isFileLoaded();
//reparse the same buffer (device is not opened again)
bool reparse();
void setCodec(const QString s);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use const references for parameters of non-basic types, not const copy

}

void CsvImportWidget::showReport() {
MessageBox::warning(this, tr("Syntax error"), tr("While parsing file...\n")
Copy link
Copy Markdown
Member

@phoerious phoerious Feb 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use the new MessageWidget instead of a MessageBox? (here and for the other two or three times you are showing error messages)

Copy link
Copy Markdown
Contributor Author

@seatedscribe seatedscribe Feb 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@phoerious It's a pretty good widget and for short messages it certainly is the right tool; in my use cases, however, I think it could bring more problems than it solves, especially with the "Show parser warnings" where it exposes a possibly long list which is handled incorrectly.
screenshot from 2017-02-21 00 58 45

I didn't dig to understand if it can handle a sidebar or can be adapted for this kind of use, but I would keep the (already large) PR codebase as it is (WRT this topic).

Copy link
Copy Markdown
Member

@phoerious phoerious Feb 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, you shouldn't show a gazillion error messages at once anyway. A message box that extends beyond the edge of the screen and therefore has no usable OK button isn't any better.
You should only show the first few message and then add " more messages" if there are any errors left.

The MessageWidget should also be at the top of the CSV widget. Right now it appears to be below the "Import CSV fields" heading and also doesn't expand horizontally. Compare its use in the DatabaseOpenWidget.

Copy link
Copy Markdown
Member

@louib louib Feb 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seatedscribe what does it look like with the same error message with a MessageBox? Maybe the MessageWidget could display a generic Invalid CSV format message when the parsing failed, and clicking Show parser warnings would show the MessageBox with all the details?

If we are to display 1 error line per line in the csv file, It might be hard to display elegantly whatever the chosen method is, so I wouldn't mind a MessageBox in this particular case.

QString groupLabel;
QStringList groupList;
bool is_root = false
, is_empty = false
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't batch initialize variables

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm? How would you put those lines?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each one with a separate type identifier.

m_skipped = skipped;
QModelIndex topLeft = createIndex(skipped,0);
QModelIndex bottomRight = createIndex(m_skipped+rowCount(), columnCount());
Q_EMIT dataChanged(topLeft, bottomRight);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We decided on using MOC keywords now instead of Q_* macros, so use emit instead of Q_EMIT

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

}

QVariant CsvParserModel::data(const QModelIndex &index, int role) const {
if ( (index.column() >= m_columnHeader.size())
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird formatting, I think this fits into one line

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

void setHeaderLabels(QStringList l);
void mapColumns(int csvColumn, int dbColumn);

virtual int rowCount(const QModelIndex &parent = QModelIndex()) const Q_DECL_OVERRIDE;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use C++11 keyword override, not Q_DECL_OVERRIDE

Copy link
Copy Markdown
Contributor Author

@seatedscribe seatedscribe Feb 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗

Copy link
Copy Markdown
Member

@phoerious phoerious Feb 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, and the virtual keyword is not needed when the overridden method is already virtual, of course.

@seatedscribe
Copy link
Copy Markdown
Contributor Author

seatedscribe commented Feb 17, 2017

@louib

The program seg faults if skip first rows is set to 1 before trying to import the csv file.

This is surprise since I tested by hand (ok, more than a year ago on the original KeepassX project, but still...) Nonetheless you are right; I'll check.

About that field, is it really a use case to skip anything else than the first (header) line? Why not just a checkbox Skip header line?

Because I don't want to make assumptions on users' habits...
Trying to be as generic as possible wont impose any forma mentis to the user; having to do with an import tool a pleasant user experience is a prerequisite if we want users to stick with KeepassXC instead of some other PM, IMHO. Generally speaking, of course. 😄

When cancelling the CSV import, the user gets asked 2 times to discard changes instead of one.

🆗

@seatedscribe
Copy link
Copy Markdown
Contributor Author

@phoerious Ehm, I may have made a mess with the merge? I wanted to rebase but it seems I am still learning by errors 😄
Should I revert or keep it as it is, adding commits onto?

@phoerious
Copy link
Copy Markdown
Member

Your call. It's your branch, nobody else has committed to. So feel free to do whatever you want.

KeePass2Writer writer;
writer.writeDatabase(&buffer, m_db);
if (writer.hasError()) {
MessageBox::warning(this, tr("Error"), tr("CSV import: writer has errors:\n")
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@phoerious Would not use messageWidget here because subsequent emit editFinished prevents the dialog to show up

QApplication::restoreOverrideCursor();

if (!result) {
MessageBox::critical(this, tr("Error"), tr("Unable to calculate master key"));
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@phoerious Would not use messageWidget here because subsequent emit importFinished prevents the dialog to show up

@seatedscribe seatedscribe force-pushed the feature/import-csv-format branch from 8a71168 to 8e8dd51 Compare March 1, 2017 00:43
@seatedscribe
Copy link
Copy Markdown
Contributor Author

@phoerious @louib This is definitive commit for you re-review.

@seatedscribe seatedscribe reopened this Mar 1, 2017
@phoerious
Copy link
Copy Markdown
Member

phoerious commented Mar 1, 2017

I only have two (three) more things I need you to fix:

You should rebase this branch on the current develop. It's quite behind the current HEAD revision and we can't merge before it's up2date with develop.

It also generates two warnings in your own code which should be fixed:

/.../keepassxc/src/core/CsvParser.cpp:318:15: warning: 'operator==' is deprecated: don't compare ints to QChars, compare them to QChar::unicode() instead [-Wdeprecated-declarations]
/.../keepassxc/src/core/CsvParser.cpp:318:15: warning: 'operator==<int>' is deprecated: don't compare ints to QChars, compare them to QChar::unicode() instead [-Wdeprecated-declarations]

and then there are a few quirks in the user interface:

screenshot_20170301_190206

The message widget has very large padding. You should not wrap the message in newlines (maybe show no success message at all?).

Also the input fields in the "Encoding" section look pretty ugly. The dropdowns have inhomogeneous heights and are lacking padding and the spin box has no border. You can easily fix that by removing the min-height of the group box and setting its vertical size policy to fixed. You should do the same to the group "Column layout" group box. While you do that, please right-align all the labels.

QApplication::restoreOverrideCursor();
if (good)
//four newline are provided to avoid resizing at every Positive/Warning switch
m_ui->messageWidget->showMessage(QString("\n\n").append(tr("CSV syntax seems in good shape !"))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the newlines here (see comment above).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

messageWidget height changes with the text, and all the rest of the window content will follow, sliding up and down. This happens every single time you change any of the Encoding parameters, so every format which is not "standard" (i.e. every format which doesn't come from a password manager) will probably require some tweaking triggering this behaviour.

Use the widget with naked messages - no newline padding - and within a couple of swirls your eyes will implore harakiri. If you can play around with this db you will understand what I mean (note the second row with a field equal to "2\"":

"Account","Login Name","Password","Web Site","Comments"
"test1","2\"","pwd","URL","some notes..."
"test","€èéç","hahaha","",""
work/group1;acc3;log3;pass3;ws3;com3;field3;lco3
work/group2/subgroup3;acc4;log4;pass4;ws4;com4;field4;lco4
fun/basket;acc5;log5;pass5;ws5;com5;field5;lco5
fun/basket;acc6;log5;pass5;ws5;com5;field5;lco5
fun/basket;acc7;log5;pass5;ws5;com5;field5;lco5
abbaz\as;accZ;log5;pass5;ws5;com5;field5;lco5

Do you think a large, reassuring green pad is so bad? Or that is it really worst than an elevator-style GUI? 😄

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I actually rebased on develop before pushing, only commits of yesterday are not included (they will be with next rollout)

  • I cannot reproduce the warnings (though I get a lot of other warnings around qhttp, Z-order assignment in EditEntryWidgetAutoType.ui and couple of 'unused result' in Application.cpp). Any hint on how can I set up to have them displayed?

  • will fix the GUI as per your comments

Copy link
Copy Markdown
Member

@phoerious phoerious Mar 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's as bad as you say, then remove the message completely. I don't really see the need for a success message. Show an error if one occurs, but we don't need an explicit success message. One can see that everything went well when the CSV contents show up correctly in the table below.

To reproduce the warnings you have to clear your built and force re-compilation of everything, I suppose. The qHttp warnings have been fixed in the release/2.1.3 branch. I think we haven't merged everything back into develop yet. Will do asap.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated develop, please rebase.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really see the need for a success message.

Me neither. But we need to handle the fact that an error message CAN happen and will dislocate the entire GUI back and forth. Is this acceptable for you? I hope not.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have an issue with an error message shifting down the GUI, since it shouldn't happen more than once. In the best case, the user will not see any message at all.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.
So I will cut out the green message and limit the warning to the first two issues. 👍

@seatedscribe
Copy link
Copy Markdown
Contributor Author

seatedscribe commented Mar 5, 2017

screenshot from 2017-03-06 00 51 23

@phoerious Rebuild is not giving me those warnings, moreover I have no apparent issues on combo paddings... Wondering what could be misbehaving. What PC are you running on?

@seatedscribe
Copy link
Copy Markdown
Contributor Author

screenshot from 2017-03-06 00 51 40

@seatedscribe seatedscribe force-pushed the feature/import-csv-format branch from 2cb3bec to 39057a6 Compare March 5, 2017 23:59
@louib
Copy link
Copy Markdown
Member

louib commented Mar 6, 2017

@seatedscribe the crash happens because of an assertion.

ASSERT: "!group->uuid().isNull()" in file src/format/KeePass2XmlWriter.cpp, line 257

Can you try with the assertions enabled?

cmake -DCMAKE_BUILD_TYPE=Debug

else
for (int i = 0; i < 2 - items; i++)
text.append(QString("\n"));
return text;
Copy link
Copy Markdown
Member

@phoerious phoerious Mar 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still getting the following warning, which is generated by this code section:

/.../keepassxc/src/gui/csvImport/CsvImportWidget.cpp: In member function ‘QString CsvImportWidget::formatStatusText() const’:
/.../keepassxc/src/gui/csvImport/CsvImportWidget.cpp:191:5: warning: this ‘else’ clause does not guard... [-Wmisleading-indentation]
     else
     ^~~~
/.../keepassxc/src/gui/csvImport/CsvImportWidget.cpp:194:9: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘else’
         return text;
         ^~~~~~

The only reason it works as intended is because it's the last statement in the function and the if block already contains a return. I would change it like this:

    if (items > 2) {
        return text.section('\n', 0, 1)
                   .append("\n[").append(QString::number(items - 2))
                   .append(tr(" more messages skipped]"));
    }

    for (int i = 0; i < 2 - items; ++i) {
        text.append(QString("\n"));
    }

    return text;

You should also replace i++ with ++i as I've done it here.

Copy link
Copy Markdown
Member

@phoerious phoerious Mar 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a different note, can items == 0 ever happen? If not, you should replace the for loop with an if condition, as it will only run when items == 1. Also do we still need this here at all? This is also only used to add newlines to error messages which isn't exactly pretty.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if it's reproducible there also, but if I skip this part altogether I get a weird layout of the messageWidget when first text showed is shorter than subsequent messages. I think the widget is expected to be sort of "one shot" and then disappear, and not supposed to update its content while not hidden:

screenshot from 2017-03-08 21 47 30

Note the lower corners are not round:
screenshot from 2017-03-08 21 50 07

So I would keep it, updated with your changes. 🆗?

Copy link
Copy Markdown
Member

@phoerious phoerious Mar 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, ok. But still fix the first issue.

@phoerious
Copy link
Copy Markdown
Member

@louib Can you verify that the crash is fixed?

@louib
Copy link
Copy Markdown
Member

louib commented Mar 14, 2017

@phoerious yup, no more seg fault when importing with asserts enabled!

@TheZ3ro
Copy link
Copy Markdown
Contributor

TheZ3ro commented Mar 15, 2017

How this behave if there are more than 6 Column (e.g. Columns for Advanced attributes ?)

@seatedscribe
Copy link
Copy Markdown
Contributor Author

seatedscribe commented Mar 15, 2017 via email

@phoerious
Copy link
Copy Markdown
Member

phoerious commented Mar 15, 2017

I think this PR is generally ready for merging. Only need to have a last look through everything.
Also the branch needs to be rebased before we can merge it, since it's a little behind develop.

@seatedscribe
Copy link
Copy Markdown
Contributor Author

seatedscribe commented Mar 15, 2017 via email

@TheZ3ro
Copy link
Copy Markdown
Contributor

TheZ3ro commented Mar 15, 2017

I just wanted to know how this perform with custom attributes, a future improvement can be the option to load custom attributes from CSV and using the column name as attributes name.

~CsvImportWidget();
void load(const QString& filename, Database* const db);

Q_SIGNALS:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be merged as is. But please run a search and replace for Q_EMIT -> emit, Q_SIGNALS -> signals and Q_SLOTS -> slots over the source code before rebasing it. We migrated to MOC keywords recently.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done presently.
Modified some other files which had Q_XYZ keywords:

src/keys/drivers/YubiKey.h
tests/TestYkChallengeResponseKey.h

<item>
<widget class="QTabWidget" name="tabWidget">
<property name="currentIndex">
<number>0</number>
Copy link
Copy Markdown
Member

@phoerious phoerious Mar 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be reverted to 0. You must have accidentally changed that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, don't know how it happened 😕

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happens when you open the UI file in the designer, click on a tab and save.

@phoerious phoerious merged commit 80fc1e5 into keepassxreboot:develop Mar 16, 2017
@seatedscribe seatedscribe deleted the feature/import-csv-format branch March 19, 2017 21:08
droidmonkey added a commit that referenced this pull request Jun 25, 2017
- Added YubiKey 2FA integration for unlocking databases [#127]
- Added TOTP support [#519]
- Added CSV import tool [#146, #490]
- Added KeePassXC CLI tool [#254]
- Added diceware password generator [#373]
- Added support for entry references [#370, #378]
- Added support for Twofish encryption [#167]
- Enabled DEP and ASLR for in-memory protection [#371]
- Enabled single instance mode [#510]
- Enabled portable mode [#645]
- Enabled database lock on screensaver and session lock [#545]
- Redesigned welcome screen with common features and recent databases [#292]
- Multiple updates to search behavior [#168, #213, #374, #471, #603, #654]
- Added auto-type fields {CLEARFIELD}, {SPACE}, {{}, {}} [#267, #427, #480]
- Fixed auto-type errors on Linux [#550]
- Prompt user prior to executing a cmd:// URL [#235]
- Entry attributes can be protected (hidden) [#220]
- Added extended ascii to password generator [#538]
- Added new database icon to toolbar [#289]
- Added context menu entry to empty recycle bin in databases [#520]
- Added "apply" button to entry and group edit windows [#624]
- Added macOS tray icon and enabled minimize on close [#583]
- Fixed issues with unclean shutdowns [#170, #580]
- Changed keyboard shortcut to create new database to CTRL+SHIFT+N [#515]
- Compare window title to entry URLs [#556]
- Implemented inline error messages [#162]
- Ignore group expansion and other minor changes when making database "dirty" [#464]
- Updated license and copyright information on souce files [#632]
- Added contributors list to about dialog [#629]
@phoerious phoerious added pr: new feature Pull request adds a new feature and removed new feature labels Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr: new feature Pull request adds a new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants