Skip to content
This repository was archived by the owner on Nov 9, 2017. It is now read-only.

Conversation

@fabpot
Copy link
Member

@fabpot fabpot commented Sep 9, 2014

Quick remainder of the problem we are facing right now with the ICU component:

When running composer install when symfony/intl is required, a trick makes Composer install either the 1.1 or 1.2 branch of the symfony/ICU component. As the decision is done at installation time, you can have problems if you deploy on a server with a different version of PHP intl (or no intl at all.) That's also a big problem for the pre-packaged Symfony projects that one can download from symfony.com as they always contain the 1.2 data (as this is what is installed on the packaging server.)

This PR proposes to switch the check at runtime via the IcuData class to fix the problem once and for all. To achieve this goal, several changes are needed:

1/ The current 1.0 branch (which contains the stubbed data for the English locale only) are moved to the symfony/intl component;
2/ The classes from all branches are moved to the symfony/intl component as well;
3/ The 1.0, 1.1, 1.2 branches are deprecated and kept only for BC reasons;
4/ The new master branch contains the data from 1.1 and 1.2. The IcuData class uses the correct version of the ICU data depending on the user PHP intl extension version.

Now, about compatibility. The PR on symfony/symfony will be done on the 2.3 branch as the problem is a real blocker for a lot of our users (and hosters), so we cannot just make it for the master branch.

Current: symfony/intl depends on symfony/icu and the right version of the data is installed depending on the intl version of your machine. So, without intl, you get the stub data, and with intl you get the 1.1 or 1.2 versions (8 or 20 Mb.)

Target: symfony/intl does not depend on symfony/icu anymore. Without an explicit dependency on symfony/icu, you will get the stubbed data (even if you have the PHP intl extension.) To get the data, one need to explicitely add the symfony/icu dependency to composer.json, which will get you the 8+20 Mb data for both ICU data formats.

As you understand, that breaks BC for people relying on the ICU data as they need to explicitely add the symfony/icu dependency (2.0 version) in their project. I think this acceptable as it fixes one of the biggest WTF problem of Symfony.

1/ and 2/ are going to be submitted as a PR to symfony/intl soon.

This PR is based on the 1.2.x branch and so it CANNOT be merged as is as it should be the new master branch of the Icu repository.

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

I forgot to mention one optimization that we might make later on: this PR packages 2 versions of the ICU data (one per format version.) The more recent version is way more compact, but it cannot be read by intl extensions, compiled against the old format (but the newer version can read the old format.)

So, if there are no significant performance gains when using the newest format, we can drop it altogether and only package the old format (that will save about 8Mb.)

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

For those reviewing this PR, the only interesting part is the changes done in the IcuData.php class: https://github.com/symfony/Icu/pull/12/files#diff-1

@webmozart
Copy link
Contributor

Looks good to me.

For DX reasons, I suggest to completely deprecate symfony/icu and create a new package symfony/intl-data instead. This will make the meaning of the package much clearer, especially now that people have to manually add it.

@webmozart
Copy link
Contributor

Furthermore, the intl/icu dependencies should be removed from composer.json. Otherwise people can't install lock-files that contain symfony/intl-data unless they have the extension loaded.

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

@webmozart I've removed the "ext-intl": "*" requirement in composer.json.
I also agree that a new repository is better.

@webmozart
Copy link
Contributor

he more recent version is way more compact, but it cannot be read by intl extensions, compiled against the old format [...] So, if there are no significant performance gains when using the newest format, we can drop it altogether and only package the old format

An alternative would be to implement the \ResourceBundle class for the new format in PHP and drop the files in the old format instead. This would make the files accessible even without the intl extension, and would reduce the package size to a minimum.

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

@webmozart What about dropping the dependency on symfony/intl as well. As we only have data, they could be used by someone else without the symfony/intl component.

@webmozart
Copy link
Contributor

@fabpot Yes, makes sense.

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

@webmozart I've added the following to the composer file:

    "suggest": {
        "symfony/intl": "To easily access the ICU data from PHP"
    },

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

@webmozart We can indeed create a new repository named symfony/intl-data but what about the namespace for the IcuData class? Do we keep the current namespace?

@stof
Copy link
Member

stof commented Sep 9, 2014

I also agree about changing the name of the package. If we want to be consistent, the class name could be Symfony\Component\IntlData\IntlData.
Btw, given that it is not really a Symfony component which is providing data, does it make sense to drop the Component part of the namespace here ?

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

@stof I agree with you: Symfony\IntlData\ for the namespace sounds good to me. IntlData for the class is fine as well.

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

And the repository would be named symfony/intl-data, right?

@stof
Copy link
Member

stof commented Sep 9, 2014

yes (and the composer package too)

@stof
Copy link
Member

stof commented Sep 9, 2014

btw, should it be intl-data or icu-data ? We are talking about ICU currently

@webmozart
Copy link
Contributor

It should be intl-data. To be 100% precise, we're talking about the CLDR data used by the ICU project, which is in turn used by PHP's intl extension. But that's useless information for our users. So intl-data should be fine.

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

@fabpot fabpot closed this Sep 9, 2014
@webmozart
Copy link
Contributor

@fabpot I thinkt that the getBundleReader() and the dependencies on Intl should be removed from IntlData.

@webmozart
Copy link
Contributor

I'm not sure whether it's good to drop the Component from the namespace for consistency reasons. Do we have any other package that lies directly under Symfony?

@stof
Copy link
Member

stof commented Sep 9, 2014

@fabpot the readme needs to be updated

@stof
Copy link
Member

stof commented Sep 9, 2014

@webmozart I'm not sure we have in the Symfony organization (which mostly contains subtree splits), but the PHP-CS-Fixer uses Symfony\CS and the prototype of the new Symfony installer uses Symfony\Toolbelt

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

symfony/symfony#11884 is the counterpart of this PR. I've submitted here now to gather feedback but as I worked on it quite a long time ago, I'm sure it's finished yet.

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

@webmozart For the Intl dep, I've created an issue on the other repository (symfony/intl-data#1.)

@fabpot
Copy link
Member Author

fabpot commented Sep 9, 2014

@stof which README file?

@stof
Copy link
Member

stof commented Sep 9, 2014

@mvrhov
Copy link

mvrhov commented Sep 9, 2014

I forgot to mention one optimization that we might make later on: this PR packages 2 versions of the ICU data (one per format version.) The more recent version is way more compact, but it cannot be read by intl extensions, compiled against the old format (but the newer version can read the old format.)

So, if there are no significant performance gains when using the newest format, we can drop it altogether and only package the old format (that will save about 8Mb.)

I can't seem to find my post where I tested all this and then described everything. But it's complicated.

AFAIR: The old format has to be generated with the old genrb as even though you specify the new genrb to generate the old format res file it doesn't do so.
Also if I remember correctly some new ICU libraries weren't able to open the old format although they should.

P.S. The data shipped here is old. As there is already ICU 53

@webmozart
Copy link
Contributor

The old format has to be generated with the old genrb as even though you specify the new genrb to generate the old format res file it doesn't do so.

That's what we do ATM.

The data shipped here is old. As there is already ICU 53

Yes, we need to update it when this bug fix is through.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants