Compare commits

...
66 Commits
Author SHA1 Message Date
Jérémy BenoistandGitHub 3042990efc Merge pull request #106 from jtojnar/encode
Fix character decoding regression when `title` precedes `meta[charset]`
2025-06-03 09:22:21 +02:00
Jan Tojnar 8b89d70b1a Fix character decoding regression when title precedes meta[charset]
Because of PHP 8.2 deprecation, in f14428e4c0, we stopped converting non-ASCII characters to HTML entities. Instead, we started to explicitly insert `meta[charset]` tag at the start of the document.

Later, we discovered that was breaking `html[lang]` so, in efbbc86df9, we made the insertion smarter. One of the improvements was that it would not insert the `meta[charset]` tag when it was already present.

That, however, broke websites that had `title` tag before `meta[charset]`. On those, libxml2 would decode the `title` contents as ISO-8859-1.

We could improve the logic (e.g. check that there is not text content before `meta[charset]`) or insert the tag unconditionally but it will probably be simplest to just go back to converting the non-ASCII characters to entities, just using non-deprecated function variant.
2025-05-28 01:10:59 +02:00
Jan Tojnar 3e9b15db46 tests: Check encoding was preserved in testHtmlLang
The fix introduced in efbbc86df9 alongside this test also manipulates `meta[charset]` but we were not checking if it does not break encoding.
2025-05-28 00:51:49 +02:00
Jérémy BenoistandGitHub 7413a38ff0 Merge pull request #104 from jtojnar/html-shadowing
Fix discarding `html[lang]`
2025-03-04 10:20:28 +01:00
Jérémy BenoistandGitHub a18cd0f2a9 Merge pull request #102 from jtojnar/local-no-domain
Do not set domainRegExp for local files
2025-03-04 10:20:15 +01:00
Jan Tojnar efbbc86df9 Fix discarding html[lang]
`DOMDocument::loadHTML` will parse HTML documents as ISO-8859-1 if there is no `meta[charset]` tag. This means that UTF-8-encoded HTML fragments such as those coming from JSON-LD `articleBody` field would be parsed with incorrect encoding.

In f14428e4c0, we tried to resolve it by putting `meta[charset]` tag at the start of the HTML fragment. Unfortunately, it turns out that causes parser to auto-insert a `html` element, losing the attributes of the original `html` tag.

Let’s try to insert the `meta[charset]` tag into the proper place in the HTML document.

We do not need to use the same trick with `JSLikeHTMLElement::__set`.
That expects smaller HTML fragments, not `html` documents, so creating `html` and `head` elements will not be a problem.
2025-03-04 01:51:34 +01:00
Jan Tojnar 541fab34a0 tests: Remove pointless debug assignment
It is unused since 8ab7d76cd5.
2025-03-03 23:55:32 +01:00
Jan Tojnar 90869d877e tests: Use ::class for DOMDocument class name
Also capitalize it properly.
2025-03-03 23:53:33 +01:00
Jan Tojnar c7208f6ad2 Do not set domainRegExp for local files
`parse_url($this->url, \PHP_URL_HOST)` will return `null` for local filesystem path.
Casting it to `string` will produce an empty regular expression,
which would match any link when computing link density.
2025-03-03 23:26:30 +01:00
Jérémy BenoistandGitHub 4258559b8a Merge pull request #100 from jtojnar/phpunit-bridge7
composer: Allow phpunit-bridge 7.0
2025-02-24 09:50:46 +01:00
Jan Tojnar 1ac761d708 composer: Allow phpunit-bridge 7.0 2025-02-24 09:47:14 +01:00
Jérémy BenoistandGitHub d3053fbce4 Merge pull request #99 from jtojnar/phpstan2
phpstan: Upgrade to version 2
2025-02-24 07:37:59 +01:00
Jan Tojnar 4c929754e9 phpstan: Upgrade to version 2
https://github.com/phpstan/phpstan/blob/2.1.x/UPGRADING.md

Required also bumping Rector since it uses PHPStan internally.
2025-02-23 02:49:08 +01:00
Jan Tojnar 1d7cdf3a12 phpstan: Use standard config path
This allows developer to create their own own config file, e.g. for setting `editorUrl`:
https://phpstan.org/user-guide/output-format#opening-file-in-an-editor
2025-02-23 02:46:57 +01:00
Jérémy BenoistandGitHub f825dcf55a Merge pull request #90 from jtojnar/foreaches
Iterate node lists with foreach
2024-10-11 08:56:35 +02:00
Jan Tojnar 9a9373de4b Iterate node lists with foreach
`DOMNodeList` implements `Traversable`.

There are some `for` loops left but we cannot simply replace those:
PHP follows the DOM specification, which requires that `NodeList`
objects in the DOM are live. As a result, any operation that removes
a node list member node from its parent (such as `removeChild`,
`replaceChild` or `appendChild`) will cause the next node
in the iterator to be skipped.

We could work around that by converting those node lists to static arrays
using `iterator_to_array` but not sure if it is worth it.
2024-10-10 09:01:45 +02:00
Jan Tojnar d454c3a462 Remove dead iteration code
This was forgotten in b580cf216d.
2024-10-10 09:01:45 +02:00
Jan Tojnar 8b1ef07401 Extract for-iterated items into variables
This simplifies the code a bit and will make it slightly easier in case we decide to switch to `foreach` iteration.
2024-10-10 09:01:45 +02:00
Jan Tojnar 5885dbbe78 Remove pointless stdClass
`DOMNode::$childNodes` always contained `DOMNodeList`.
2024-10-10 09:01:45 +02:00
Jérémy BenoistandGitHub 6947999782 Merge pull request #92 from jtojnar/ci-fix
ci: Fix & add PHP 8.4
2024-10-10 08:59:28 +02:00
Jan Tojnar da755013aa Remove extra set_error_handler callback argument
It is unused and would cause an error on PHP ≥ 8.0:
https://www.php.net/manual/en/function.set-error-handler.php#refsect1-function.set-error-handler-parameters

Not sure if the handler is even necessary – it was introduced in 175196d6c2 but I did not manage to reproduce the original error (Entity 'nbsp' not defined). It was probably fixed by f2a43b476c.
2024-10-10 08:52:28 +02:00
Jan Tojnar 5b9551d1e3 ci: Add PHP 8.4
PHP 8.4 is in beta, with final version scheduled for November so it is time to start testing it.
2024-10-10 01:27:59 +02:00
Jan Tojnar c7b10dcc45 Avoid E_STRICT constant
It will be deprecated in PHP 8.4 and it is meaningless nowadays anyway:
https://wiki.php.net/rfc/deprecations_php_8_4#remove_e_strict_error_level_and_deprecate_e_strict_constant

The use of the constant was introduced in 175196d6c2.
2024-10-10 01:27:59 +02:00
Jan Tojnar 80adfe870b Fix coding style
With php-cs-fixer 3.64.0, the `native_function_invocation` rule no longer passed.
2024-10-10 01:01:55 +02:00
Jérémy BenoistandGitHub cb6b6ac577 Merge pull request #88 from jtojnar/has-single-fix 2024-03-19 06:02:54 +01:00
Jan Tojnar 677f3f096e Fix hasSingleTagInsideElement method
It would fail for e.g. `<div> <p>foo</p> </div>`.

mozilla/readability uses children for the tag lookup, which return only elements.
PHP does not have children property so b580cf216d
mistakenly used `childNodes` instead, but that can return any node type.

Let’s filter the children ourselves.

Also add comments from mozilla/readability’s `_hasSingleTagInsideElement`.
2024-03-18 23:01:43 +01:00
Jérémy BenoistandGitHub 29122763db Merge pull request #89 from jtojnar/php74
Require PHP 7.4
2024-03-18 09:18:11 +01:00
Jan TojnarandJérémy Benoist 89d3b74259 Rectorize to PHP 7.4
Switches to short anonymous function syntax.
2024-03-18 09:16:43 +01:00
Jan TojnarandJérémy Benoist e792644fe8 Drop PHP < 7.4 support
This will allow us to use flexible heredocs in test,
as well as typed properties and other goodies.

https://www.php.net/releases/7_3_0.php
https://www.php.net/releases/7_4_0.php
2024-03-18 09:16:43 +01:00
Jan TojnarandJérémy Benoist 648d8c605b Update coding style for upcoming PHP-CS-Fixer changes
Once we bump minimum PHP version, we will get newer PHP-CS-Fixer,
which will try to apply this cleanups.

Also manually tweak anonymous functions so that they are cleanly formatted
once we switch to `fn` syntax.
2024-03-18 09:16:43 +01:00
Jérémy BenoistandGitHub f28191a728 Merge pull request #86 from jtojnar/ci-bump
ci: Update actions
2024-03-18 09:12:18 +01:00
Jan Tojnar 2103853a1b ci: Bump coveralls to 2.7.0
- Fixes PHP 8 support https://github.com/php-coveralls/php-coveralls/releases/tag/v2.4.3
2024-03-16 22:30:49 +01:00
Jan Tojnar 7f4c6cfcbd ci: Update actions
Mostly just of nodejs bump:

- https://github.com/actions/checkout/releases/tag/v4.0.0
- https://github.com/ramsey/composer-install/releases/tag/3.0.0
2024-03-16 16:01:16 +01:00
Jérémy BenoistandGitHub 38870cdff1 Merge pull request #80 from jtojnar/stricter
Fix some CI issues
2023-04-03 14:47:32 +02:00
Jan Tojnar 9bdd3b6b2e ci: Add PHP 8.2 and 8.3 2023-03-31 05:26:07 +02:00
Jan Tojnar f14428e4c0 Do not use mb_convert_encoding with HTML-ENTITIES as target encoding
This is deprecated since PHP 8.2:

    Deprecated: mb_convert_encoding(): Handling HTML entities via mbstring is deprecated; use htmlspecialchars, htmlentities, or mb_encode_numericentity/mb_decode_numericentity instead

It was used because `DOMDocument`, which uses libxml2 internally, will parse the HTML as ISO-8859-1, unless the document contains an XML encoding declaration or HTML meta tag setting character set.
Since first such element wins, putting the `meta[charset]` up front will ensure the parser uses the correct encoding, even if the document contains incorrect meta tag (e.g. when the document is converted to UTF-8 without also updating the metadata by the software passing it to Readability).

https://stackoverflow.com/a/39148511/160386
2023-03-31 05:26:07 +02:00
Jan Tojnar 23f824a1ce tests: Fix “THE ERROR HANDLER HAS CHANGED!” 2023-03-31 03:19:22 +02:00
Jan Tojnar 2a57124528 composer: upgrade rector 2023-03-31 03:19:22 +02:00
Jan Tojnar 0975574bdb Rector: Upgrade configuration 2023-03-31 03:19:22 +02:00
Jan Tojnar 9ed89bde92 Fix PHP-Cs-Fixer changes
1) src/Readability.php (braces, no_unneeded_control_parentheses, single_line_comment_spacing, global_namespace_import, no_unused_imports, phpdoc_align)
   2) src/JSLikeHTMLElement.php (phpdoc_separation)

Switch code blocks to Markdown syntax to work around `phpdoc_separation`, ApiGen uses Markdown these days anyway.
2023-03-31 03:14:00 +02:00
Jan Tojnar 2c6c6d5987 PHPStan: Use stable PHPUnit path
phpunit-bridge will create a symlink.
2023-03-31 03:14:00 +02:00
Jan Tojnar c5407ec07c composer: Add scripts for development 2023-03-31 03:14:00 +02:00
Jérémy BenoistandGitHub 7cd8476d38 Merge pull request #79 from j0k3r/fix/psr-log-2-3
Allow `psr/log` 2.0 & 3.0
2022-10-17 22:44:36 +02:00
Jeremy Benoist 82083c872b Allow psr/log 2.0 & 3.0 2022-10-17 22:42:47 +02:00
Kevin DecherfandJeremy Benoist 6689f19956 Strip script and style tags through ::clean() method instead of preg_replace
Huge tags can lead to a failure of preg_replace, thus erasing the whole
fetched content.

Fixes https://github.com/wallabag/wallabag/issues/5847

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-06-13 09:13:23 +02:00
Jérémy BenoistandGitHub 0c0653dad6 Merge pull request #73 from Kdecherf/fix/impr
Fix `isPhrasingContent` conditions, text node replacement
2022-02-16 00:03:37 +01:00
Kevin Decherf 2ab87d7445 Fix isPhrasingContent conditions, text node replacement
It also disables reverting forced paragraph elements as it can break
layouts or corrupt content.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-02-15 20:53:28 +01:00
Jérémy BenoistandGitHub 8af69ad68c Merge pull request #71 from j0k3r/feature/enable-rector
Add Rector
2022-02-04 12:15:57 +01:00
Jeremy Benoist c2a1639b34 Add Rector 2022-02-04 12:13:37 +01:00
Jérémy BenoistandGitHub ccf1b336c5 Merge pull request #64 from Kdecherf/improvements 2022-02-04 05:27:57 +01:00
Kevin Decherf a44c4e5482 Add routine to remove invisible nodes
Readability was previously removing (was trying to actually, see next
section) invisible nodes using a pattern from `unlikelyCandidates`. This
was quite hacky and was removed during a backport of logics from
mozilla/readability. There is still a need to remove them so here we
are. We still use a pattern but specifically against the style
attribute. We also remove nodes with the attribute `hidden`.

The clean feature of tidy actually replaces inline style attributes
with css classes thus preventing readability to detect invisible nodes,
see https://github.com/htacg/tidy-html5/blob/5.6.0/src/clean.c#L1488
We therefore set clean configuration to false.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-02-04 01:23:11 +01:00
Kevin Decherf b580cf216d Backport some logics from mozilla/readability
This change backports several things from mozilla/readability:

- Add child score to all ancestors instead of the first parent only
- Check 5 top candidates and try to find alternative candidates within
  ancestors, this can help to find a better parent and grab more content
- Reduce patterns from `unlikelyCandidates` to the one used by Mozilla
  as ours tend to remove useful nodes
- Score headers (h2 to h6) by default in addition to div, p, td and
  section

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-02-04 01:05:13 +01:00
Jérémy BenoistandGitHub 2e9349f076 Merge pull request #69 from j0k3r/feature/php-7.2
Require PHP >= 7.2
2022-02-02 12:53:28 +01:00
Jeremy Benoist c4bba53dbe Remove Scrutinizer 2022-02-02 12:52:12 +01:00
Jeremy Benoist 66215a6c80 Require PHP >= 7.2
- remove test on Composer v1
- remove deprecated function
- move `loadHtml()` into `init()` instead of `__construct`

Kinda prepare 2.0 version :)
2022-02-02 12:44:24 +01:00
Jérémy BenoistandGitHub b1a20a9575 Merge pull request #68 from open-source-contributions/master
Using assertSame to make assertion equal strict
2021-12-17 12:17:44 +01:00
peter279k 97c02e8ad4 Using assertSame to make assertion equal strict 2021-12-17 19:09:36 +08:00
Jérémy BenoistandGitHub c506b7ebd7 Merge pull request #67 from j0k3r/fix/psr-log-void
Fix deprecated message
2021-11-29 21:06:47 +01:00
Jeremy Benoist d0af21814a Ditch assertContains & assertNotContains 2021-11-29 21:04:56 +01:00
Jeremy Benoist 5b1eba79bd Test on PHP 8 & drop Travis 2021-11-29 21:00:36 +01:00
Jérémy BenoistandGitHub fabf096ce6 Fix deprecated message
> Method "Psr\Log\LoggerAwareInterface::setLogger()" might add "void" as a native return type declaration in the future. Do the same in implementation "Readability\Readability" now to avoid errors or add an explicit @return annotation to suppress this message.
2021-11-29 20:50:10 +01:00
Jérémy BenoistandGitHub 8ce1663238 Merge pull request #66 from Kdecherf/fix/figure 2021-10-29 16:24:41 +02:00
Kevin Decherf eb72a315c4 Clean empty figure tags without ending
See 'Tag omission' https://developer.mozilla.org/en-US/docs/Web/HTML/Element/figure

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2021-10-29 15:36:19 +02:00
Jérémy BenoistandGitHub d5330a9c28 Merge pull request #65 from j0k3r/fix/php-cs
Fix PHPCS config file
2021-10-04 11:42:34 +02:00
Jeremy Benoist 635f8963dc Fix Travis & PHPStan 2021-10-04 11:37:33 +02:00
Jeremy Benoist 19b2a25d96 Fix PHPCS config file 2021-10-04 11:32:34 +02:00
15 changed files with 828 additions and 727 deletions
+1
View File
@@ -6,5 +6,6 @@
/.php_cs export-ignore
/phpunit.xml.dist export-ignore
/phpstan.neon export-ignore
/rector.php export-ignore
/.github export-ignore
/tests export-ignore
+5 -11
View File
@@ -8,38 +8,32 @@ on:
branches:
- master
env:
SYMFONY_PHPUNIT_VERSION: 7.5
jobs:
coding-standards:
name: "CS Fixer & PHPStan"
runs-on: "ubuntu-18.04"
runs-on: "ubuntu-22.04"
strategy:
matrix:
php:
- "7.3"
- "7.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v2"
uses: "actions/checkout@v4"
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
coverage: "none"
php-version: "${{ matrix.php }}"
tools: cs2pr, composer:v1
tools: cs2pr, composer:v2
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Add PHPStan"
run: "composer require phpstan/phpstan phpstan/phpstan-phpunit --dev --no-progress --no-suggest"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1"
uses: "ramsey/composer-install@v3"
with:
composer-options: "--optimize-autoloader --prefer-dist"
+99 -151
View File
@@ -14,163 +14,21 @@ env:
jobs:
phpunit:
name: "PHPUnit (PHP ${{ matrix.php }})"
runs-on: "ubuntu-18.04"
strategy:
matrix:
php:
- "5.6"
- "7.0"
- "7.1"
- "7.2"
- "7.3"
- "7.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v2"
with:
fetch-depth: 2
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "none"
tools: composer:v1
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Force PHPUnit version"
if: matrix.php >= '7.2'
run: "echo $SYMFONY_PHPUNIT_VERSION"
env:
SYMFONY_PHPUNIT_VERSION: 7.5
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1"
with:
composer-options: "--optimize-autoloader --prefer-dist"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"
phpunit-coverage:
name: "PHPUnit coverage (PHP ${{ matrix.php }})"
runs-on: "ubuntu-18.04"
runs-on: "ubuntu-22.04"
strategy:
matrix:
php:
- "7.4"
- "8.0"
- "8.1"
- "8.2"
- "8.3"
- "8.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v2"
with:
fetch-depth: 2
- name: "Install PHP with Xdebug"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "xdebug"
tools: composer:v1
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1"
with:
composer-options: "--optimize-autoloader --prefer-dist"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit (with coverage)"
run: "php vendor/bin/simple-phpunit -v --coverage-clover build/logs/clover.xml"
env:
SYMFONY_PHPUNIT_VERSION: 7.5
- name: "Retrieve Coveralls phar"
run: "wget https://github.com/php-coveralls/php-coveralls/releases/download/v2.4.2/php-coveralls.phar"
- name: "Enable Coveralls phar"
run: "chmod +x php-coveralls.phar"
- name: "Upload to Coveralls"
run: "php php-coveralls.phar -v -x build/logs/clover.xml"
env:
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
phpunit-lowest:
name: "PHPUnit lowest deps (PHP ${{ matrix.php }})"
runs-on: "ubuntu-18.04"
strategy:
matrix:
php:
- "7.2"
steps:
- name: "Checkout"
uses: "actions/checkout@v2"
with:
fetch-depth: 2
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "none"
tools: composer:v1
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1"
with:
composer-options: "--optimize-autoloader --prefer-dist"
dependency-versions: "lowest"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"
env:
SYMFONY_PHPUNIT_VERSION: 7.5
phpunit-composerv2:
name: "PHPUnit with Composer v2 (PHP ${{ matrix.php }})"
runs-on: "ubuntu-18.04"
strategy:
matrix:
php:
- "7.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v2"
uses: "actions/checkout@v4"
with:
fetch-depth: 2
@@ -189,7 +47,7 @@ jobs:
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1"
uses: "ramsey/composer-install@v3"
with:
composer-options: "--optimize-autoloader --prefer-dist"
@@ -198,5 +56,95 @@ jobs:
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"
phpunit-coverage:
name: "PHPUnit coverage (PHP ${{ matrix.php }})"
runs-on: "ubuntu-22.04"
strategy:
matrix:
php:
- "8.0"
steps:
- name: "Checkout"
uses: "actions/checkout@v4"
with:
fetch-depth: 2
- name: "Install PHP with Xdebug"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "xdebug"
tools: composer:v2
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
SYMFONY_PHPUNIT_VERSION: 7.5
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v3"
with:
composer-options: "--optimize-autoloader --prefer-dist"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit (with coverage)"
run: "php vendor/bin/simple-phpunit -v --coverage-clover build/logs/clover.xml"
- name: "Retrieve Coveralls phar"
run: "wget https://github.com/php-coveralls/php-coveralls/releases/download/v2.7.0/php-coveralls.phar"
- name: "Enable Coveralls phar"
run: "chmod +x php-coveralls.phar"
- name: "Upload to Coveralls"
run: "php php-coveralls.phar -v -x build/logs/clover.xml"
env:
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
phpunit-lowest:
name: "PHPUnit lowest deps (PHP ${{ matrix.php }})"
runs-on: "ubuntu-22.04"
strategy:
matrix:
php:
- "7.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v4"
with:
fetch-depth: 2
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "none"
tools: composer:v2
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v3"
with:
composer-options: "--optimize-autoloader --prefer-dist"
dependency-versions: "lowest"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"
+2
View File
@@ -2,4 +2,6 @@ vendor/
coverage/
composer.lock
.php_cs.cache
.php-cs-fixer.cache
.phpunit.result.cache
phpstan.neon
+12 -7
View File
@@ -1,6 +1,11 @@
<?php
return PhpCsFixer\Config::create()
$finder = (new PhpCsFixer\Finder())
->in(__DIR__)
->exclude(['vendor', 'var', 'web'])
;
return (new PhpCsFixer\Config())
->setUsingCache(true)
->setRiskyAllowed(true)
->setRules([
@@ -9,7 +14,7 @@ return PhpCsFixer\Config::create()
'array_syntax' => ['syntax' => 'short'],
'combine_consecutive_unsets' => true,
'heredoc_to_nowdoc' => true,
'no_extra_consecutive_blank_lines' => ['break', 'continue', 'extra', 'return', 'throw', 'use', 'parenthesis_brace_block', 'square_brace_block', 'curly_brace_block'],
'no_extra_blank_lines' => ['tokens' => ['break', 'continue', 'extra', 'return', 'throw', 'use', 'parenthesis_brace_block', 'square_brace_block', 'curly_brace_block']],
'no_unreachable_default_argument_value' => true,
'no_useless_else' => true,
'no_useless_return' => true,
@@ -21,10 +26,10 @@ return PhpCsFixer\Config::create()
'strict_comparison' => true,
'strict_param' => true,
'concat_space' => ['spacing' => 'one'],
// Pulled in by @Symfony:risky but we still support PHP 7.4
'modernize_strpos' => false,
// Pulled in by @Symfony, we cannot add property types until we bump PHP to ≥ 7.4
'no_null_property_initialization' => false,
])
->setFinder(
PhpCsFixer\Finder::create()
->exclude(['vendor'])
->in(__DIR__)
)
->setFinder($finder)
;
-12
View File
@@ -1,12 +0,0 @@
tools:
external_code_coverage: false
build:
nodes:
analysis:
tests:
override:
- php-scrutinizer-run
environment:
php:
version: 7.2
-34
View File
@@ -1,34 +0,0 @@
dist: xenial
os: linux
language: php
# avoid double Travis build when the PR is created on upstream
if: |
type = pull_request OR \
branch = master
jobs:
include:
- php: 8.0
env: SYMFONY_PHPUNIT_VERSION=7.5
fast_finish: true
allow_failures:
- php: 8.0
cache:
directories:
- $HOME/.composer/cache/files
before_install:
- if [ -n "$GH_TOKEN" ]; then composer config github-oauth.github.com ${GH_TOKEN}; fi;
install:
- if [ "$CS_FIXER" != "run" ]; then composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update ; fi;
- composer update --prefer-dist --no-progress --no-suggest -o $
- php vendor/bin/simple-phpunit install
before_script:
- mkdir -p build/logs
script:
- php vendor/bin/simple-phpunit -v
+1 -2
View File
@@ -1,12 +1,11 @@
# Readability
![CI](https://github.com/j0k3r/php-readability/workflows/CI/badge.svg)
[![Build Status](https://travis-ci.org/j0k3r/php-readability.svg?branch=master)](https://travis-ci.org/j0k3r/php-readability)
[![Coverage Status](https://coveralls.io/repos/j0k3r/php-readability/badge.svg?branch=master&service=github)](https://coveralls.io/github/j0k3r/php-readability/?branch=master)
[![Total Downloads](https://poser.pugx.org/j0k3r/php-readability/downloads)](https://packagist.org/packages/j0k3r/php-readability)
[![License](https://poser.pugx.org/j0k3r/php-readability/license)](https://packagist.org/packages/j0k3r/php-readability)
This is an extract of the Readability class from this [full-text-rss](https://github.com/Dither/full-text-rss) fork. It can be defined as a better version of the original [php-readability](https://bitbucket.org/fivefilters/php-readability/overview).
This is an extract of the Readability class from this [full-text-rss](https://github.com/Dither/full-text-rss) fork. It can be defined as a better version of the original [php-readability](https://bitbucket.org/fivefilters/php-readability).
## Differences
+13 -4
View File
@@ -24,15 +24,18 @@
"role": "Developer (original JS version)"
}],
"require": {
"php": ">=5.6.0",
"php": ">=7.4.0",
"ext-mbstring": "*",
"psr/log": "^1.0",
"psr/log": "^1.0.1 || ^2.0 || ^3.0",
"masterminds/html5": "^2.7"
},
"require-dev": {
"friendsofphp/php-cs-fixer": "^2.14",
"friendsofphp/php-cs-fixer": "^3.0",
"monolog/monolog": "^1.24|^2.1",
"symfony/phpunit-bridge": "^4.2.3"
"symfony/phpunit-bridge": "^4.4|^5.3|^6.0|^7.0",
"phpstan/phpstan": "^2.0",
"phpstan/phpstan-phpunit": "^2.0",
"rector/rector": "^2.0.0"
},
"suggest": {
"ext-tidy": "Used to clean up given HTML and to avoid problems with bad HTML structure."
@@ -42,5 +45,11 @@
},
"autoload-dev": {
"psr-4": { "Tests\\Readability\\": "tests/" }
},
"scripts": {
"fix": "php-cs-fixer fix --verbose --diff",
"phpstan": "phpstan analyze --memory-limit 512M",
"rector": "rector process",
"test": "simple-phpunit -v"
}
}
+1 -1
View File
@@ -6,7 +6,7 @@ parameters:
# https://github.com/phpstan/phpstan/issues/694#issuecomment-350724288
bootstrapFiles:
- vendor/bin/.phpunit/phpunit-7.5-0/vendor/autoload.php
- vendor/bin/.phpunit/phpunit/vendor/autoload.php
includes:
- vendor/phpstan/phpstan-phpunit/extension.neon
+1 -1
View File
@@ -11,7 +11,7 @@
>
<testsuites>
<testsuite name="Readability Test Suite">
<testsuite name="Readability">
<directory>./tests/</directory>
</testsuite>
</testsuites>
+18
View File
@@ -0,0 +1,18 @@
<?php
declare(strict_types=1);
use Rector\Config\RectorConfig;
use Rector\Set\ValueObject\LevelSetList;
return RectorConfig::configure()
->withPaths([
__DIR__ . '/src',
__DIR__ . '/tests',
])
->withBootstrapFiles([
__DIR__ . '/vendor/bin/.phpunit/phpunit/vendor/autoload.php',
__DIR__ . '/vendor/autoload.php',
])
->withSets([LevelSetList::UP_TO_PHP_74])
;
+5 -6
View File
@@ -39,9 +39,9 @@ class JSLikeHTMLElement extends \DOMElement
/**
* Used for setting innerHTML like it's done in JavaScript:.
*
* @code
* ```php
* $div->innerHTML = '<h2>Chapter 2</h2><p>The story begins...</p>';
* @endcode
* ```
*/
public function __set($name, $value)
{
@@ -79,14 +79,13 @@ class JSLikeHTMLElement extends \DOMElement
} else {
// $value is probably ill-formed
$f = new \DOMDocument();
$value = mb_convert_encoding($value, 'HTML-ENTITIES', 'UTF-8');
// Using <htmlfragment> will generate a warning, but so will bad HTML
// (and by this point, bad HTML is what we've got).
// We use it (and suppress the warning) because an HTML fragment will
// be wrapped around <html><body> tags which we don't really want to keep.
// Note: despite the warning, if loadHTML succeeds it will return true.
$result = $f->loadHTML('<htmlfragment>' . $value . '</htmlfragment>');
$result = $f->loadHTML('<meta charset="utf-8"><htmlfragment>' . $value . '</htmlfragment>');
if ($result) {
$import = $f->getElementsByTagName('htmlfragment')->item(0);
@@ -105,9 +104,9 @@ class JSLikeHTMLElement extends \DOMElement
/**
* Used for getting innerHTML like it's done in JavaScript:.
*
* @code
* ```php
* $string = $div->innerHTML;
* @endcode
* ```
*/
public function __get($name)
{
+385 -304
View File
File diff suppressed because it is too large Load Diff
+285 -194
View File
@@ -4,379 +4,377 @@ namespace Tests\Readability;
use Monolog\Handler\TestHandler;
use Monolog\Logger;
use Psr\Log\LoggerInterface;
use Readability\JSLikeHTMLElement;
use Readability\Readability;
class ReadabilityTest extends \PHPUnit\Framework\TestCase
{
/** @var TestHandler */
public $logHandler;
/** @var LoggerInterface */
public $logger;
/**
* @requires extension tidy
*/
public function testConstructDefault()
public function testConstructDefault(): void
{
$readability = $this->getReadability('');
$readability->init();
$this->assertNull($readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertInstanceOf(\DOMDocument::class, $readability->dom);
}
public function testConstructHtml5Parser()
public function testConstructHtml5Parser(): void
{
$readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'html5lib');
$readability->init();
$this->assertEquals('http://0.0.0.0', $readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertEquals('<html/>', $readability->original_html);
$this->assertSame('http://0.0.0.0', $readability->url);
$this->assertInstanceOf(\DOMDocument::class, $readability->dom);
$this->assertSame('<html/>', $readability->original_html);
}
/**
* @requires extension tidy
*/
public function testConstructSimple()
public function testConstructSimple(): void
{
$readability = $this->getReadability('<html/>', 'http://0.0.0.0');
$readability->init();
$this->assertEquals('http://0.0.0.0', $readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertEquals('<html/>', $readability->original_html);
$this->assertSame('http://0.0.0.0', $readability->url);
$this->assertInstanceOf(\DOMDocument::class, $readability->dom);
$this->assertSame('<html/>', $readability->original_html);
$this->assertTrue($readability->tidied);
}
public function testConstructDefaultWithoutTidy()
public function testConstructDefaultWithoutTidy(): void
{
$readability = $this->getReadability('', null, 'libxml', false);
$readability->init();
$this->assertNull($readability->url);
$this->assertEquals('', $readability->original_html);
$this->assertSame('', $readability->original_html);
$this->assertFalse($readability->tidied);
$this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertInstanceOf(\DOMDocument::class, $readability->dom);
}
public function testConstructSimpleWithoutTidy()
public function testConstructSimpleWithoutTidy(): void
{
$readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'libxml', false);
$readability->init();
$this->assertEquals('http://0.0.0.0', $readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertEquals('<html/>', $readability->original_html);
$this->assertSame('http://0.0.0.0', $readability->url);
$this->assertInstanceOf(\DOMDocument::class, $readability->dom);
$this->assertSame('<html/>', $readability->original_html);
$this->assertFalse($readability->tidied);
}
public function testInitNoContent()
public function testInitNoContent(): void
{
$readability = $this->getReadability('<html/>', 'http://0.0.0.0');
$res = $readability->init();
$this->assertFalse($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('Sorry, Readability was unable to parse this page for content.', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('Sorry, Readability was unable to parse this page for content.', $readability->getContent()->getInnerHtml());
}
public function testInitP()
public function testInitP(): void
{
$readability = $this->getReadability(str_repeat('<p>This is the awesome content :)</p>', 7), 'http://0.0.0.0');
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is the awesome content :)', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is the awesome content :)', $readability->getContent()->getInnerHtml());
}
public function testInitDivP()
public function testInitDivP(): void
{
$readability = $this->getReadability('<div>' . str_repeat('<p>This is the awesome content :)</p>', 7) . '</div>', 'http://0.0.0.0');
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is the awesome content :)', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is the awesome content :)', $readability->getContent()->getInnerHtml());
}
public function testInitDiv()
public function testInitDiv(): void
{
$readability = $this->getReadability('<div>' . str_repeat('This is the awesome content :)', 7) . '</div>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is the awesome content :)', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is the awesome content :)', $readability->getContent()->getInnerHtml());
}
public function testWithFootnotes()
public function testWithFootnotes(): void
{
$readability = $this->getReadability('<div>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '</div>', 'http://0.0.0.0');
$readability->debug = true;
$readability->convertLinksToFootnotes = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('readabilityFootnoteLink', $readability->getContent()->getInnerHtml());
$this->assertContains('readabilityLink-3', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('readabilityFootnoteLink', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('readabilityLink-3', $readability->getContent()->getInnerHtml());
}
public function testStandardClean()
public function testStandardClean(): void
{
$readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<a href="#nofollow" rel="nofollow">will NOT be removed</a></div>', 'http://0.0.0.0');
$readability->debug = true;
$readability->lightClean = false;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('will NOT be removed', $readability->getContent()->getInnerHtml());
$this->assertNotContains('<h2>', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('will NOT be removed', $readability->getContent()->getInnerHtml());
$this->assertStringNotContainsString('<h2>', $readability->getContent()->getInnerHtml());
}
public function testWithIframe()
public function testWithIframe(): void
{
$readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe><iframe>http://soundcloud.com/test</iframe></p></div>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('nofollow', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('nofollow', $readability->getContent()->getInnerHtml());
}
public function testWithArticle()
public function testWithArticle(): void
{
$readability = $this->getReadability('<article><p>' . str_repeat('This is an awesome text with some links, here there are: the awesome', 20) . '</p><p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('nofollow', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('nofollow', $readability->getContent()->getInnerHtml());
}
public function testWithAside()
public function testWithAside(): void
{
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<footer><aside>' . str_repeat('<p>This is an awesome text with some links, here there are</p>', 8) . '</aside></footer></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('<aside>', $readability->getContent()->getInnerHtml());
$this->assertContains('<footer readability="5"/>', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringNotContainsString('<aside>', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('<footer readability="9"/>', $readability->getContent()->getInnerHtml());
}
public function testWithClasses()
public function testWithClasses(): void
{
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text should be removed', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringNotContainsString('This text should be removed', $readability->getContent()->getInnerHtml());
}
public function testWithClassesWithoutLightClean()
public function testWithClassesWithoutLightClean(): void
{
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0');
$readability->debug = true;
$readability->lightClean = false;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text should be removed', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringNotContainsString('This text should be removed', $readability->getContent()->getInnerHtml());
}
public function testWithTd()
public function testWithTd(): void
{
$readability = $this->getReadability('<table><tr>' . str_repeat('<td><p>This is an awesome text with some links, here there are the awesome</td>', 7) . '</tr></table>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
}
public function testWithSameClasses()
public function testWithSameClasses(): void
{
$readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<div class="awesomecontent">This text is also an awesome text and you should know that !</div></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
}
public function testWithScript()
public function testWithScript(): void
{
$readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p><script>This text is also an awesome text and you should know that !</script></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
}
public function testTitle()
public function testTitle(): void
{
$readability = $this->getReadability('<title>this is my title</title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEquals('this is my title', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertSame('this is my title', $readability->getTitle()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
}
public function testTitleWithDash()
public function testTitleWithDash(): void
{
$readability = $this->getReadability('<title> title2 - title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEquals('title2 - title3', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertSame('title2 - title3', $readability->getTitle()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
}
public function testTitleWithDoubleDot()
public function testTitleWithDoubleDot(): void
{
$readability = $this->getReadability('<title> title2 : title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEquals('title2 : title3', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertSame('title2 : title3', $readability->getTitle()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
}
public function testTitleTooShortUseH1()
public function testTitleTooShortUseH1(): void
{
$readability = $this->getReadability('<title>too short</title><h1>this is my h1 title !</h1><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEquals('this is my h1 title !', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertSame('this is my h1 title !', $readability->getTitle()->getInnerHtml());
$this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
}
// dummy function to be used to the next test
public function error2Exception($code, $string, $file, $line, $context)
public function testAutoClosingIframeNotThrowingException(): void
{
throw new \Exception($string, $code);
}
$oldErrorReporting = error_reporting(\E_ALL);
$oldDisplayErrors = ini_set('display_errors', '1');
// dummy function to be used to the next test
set_error_handler(function (int $errno, string $errstr, string $errfile, int $errline) {
throw new \Exception($errstr, $errno);
});
public function testAutoClosingIframeNotThrowingException()
{
error_reporting(\E_ALL | \E_STRICT);
ini_set('display_errors', true);
set_error_handler([$this, 'error2Exception'], \E_ALL | \E_STRICT);
try {
$data = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="ru-RU" prefix="og: http://ogp.me/ns#">
$data = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="ru-RU" prefix="og: http://ogp.me/ns#">
<head profile="http://gmpg.org/xfn/11">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<head profile="http://gmpg.org/xfn/11">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
</head>
<body class="single single-post postid-22030 single-format-standard">
<div id="wrapper">
<div id="content">
<div class="post-22030 post type-post status-publish format-standard has-post-thumbnail hentry category-video category-reviews tag-193" id="post-22030">
<h1>3D Touch &#8212; будущее мобильных игр</h1>
<div class="postdate">Автор: <strong>Сергей Пак</strong> | Просмотров: 1363 | Опубликовано: 14 сентября 2015 </div>
<div class="entry">
<p>Компания Apple представила новую технологию 3D Touch, которая является прямым потомком более ранней версии Force Touch &#8212; последняя, напомним, используется сейчас в трекпадах Macbook Pro и Macbook 2015. Теперь управлять устройством стало в разы проще, и Force Touch открывает перед пользователями новые возможности, но при этом 3D Touch &#8212; это про другое. Дело в том, что теперь и на мобильных устройствах интерфейс будет постепенно меняться, кардинальные перемены ждут мобильный гейминг, потому что здесь разработчики действительно могут разгуляться.<span id="more-22030"></span></p>
<p><iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p>
<p>Итак, просто представьте себе, что iPhone 6S &#8212; это, по большому счету, отличная игровая приставка, которую вы носите с собой, а еще она может выдавать невероятной красоты картинку. Но проблема заключается, пожалуй, в том, что управлять персонажем в играх довольно трудно &#8212; он неповоротлив, обладает заторможенной реакцией, а игровой клиент зачастую требует перегруза интерфейса для того, чтобы обеспечить максимально большое количество возможностей. Благодаря трехуровневому нажатию можно избавиться от лишних кнопок и обеспечить более качественный обзор местности, и при этом пользователь будет закрывать пальцами минимальное пространство.</p>
</head>
<body class="single single-post postid-22030 single-format-standard">
<div id="wrapper">
<div id="content">
<div class="post-22030 post type-post status-publish format-standard has-post-thumbnail hentry category-video category-reviews tag-193" id="post-22030">
<h1>3D Touch &#8212; будущее мобильных игр</h1>
<div class="postdate">Автор: <strong>Сергей Пак</strong> | Просмотров: 1363 | Опубликовано: 14 сентября 2015 </div>
<div class="entry">
<p>Компания Apple представила новую технологию 3D Touch, которая является прямым потомком более ранней версии Force Touch &#8212; последняя, напомним, используется сейчас в трекпадах Macbook Pro и Macbook 2015. Теперь управлять устройством стало в разы проще, и Force Touch открывает перед пользователями новые возможности, но при этом 3D Touch &#8212; это про другое. Дело в том, что теперь и на мобильных устройствах интерфейс будет постепенно меняться, кардинальные перемены ждут мобильный гейминг, потому что здесь разработчики действительно могут разгуляться.<span id="more-22030"></span></p>
<p><iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p>
<p>Итак, просто представьте себе, что iPhone 6S &#8212; это, по большому счету, отличная игровая приставка, которую вы носите с собой, а еще она может выдавать невероятной красоты картинку. Но проблема заключается, пожалуй, в том, что управлять персонажем в играх довольно трудно &#8212; он неповоротлив, обладает заторможенной реакцией, а игровой клиент зачастую требует перегруза интерфейса для того, чтобы обеспечить максимально большое количество возможностей. Благодаря трехуровневому нажатию можно избавиться от лишних кнопок и обеспечить более качественный обзор местности, и при этом пользователь будет закрывать пальцами минимальное пространство.</p>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>';
</body>
</html>';
$readability = $this->getReadability($data, 'http://iosgames.ru/?p=22030');
$readability->debug = true;
$readability = $this->getReadability($data, 'http://iosgames.ru/?p=22030');
$res = $readability->init();
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"> </iframe>', $readability->getContent()->getInnerHtml());
$this->assertContains('3D Touch', $readability->getTitle()->getInnerHtml());
$this->assertTrue($res);
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertStringContainsString('<iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"> </iframe>', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('3D Touch', $readability->getTitle()->getInnerHtml());
} finally {
restore_error_handler();
if (false !== $oldDisplayErrors) {
ini_set('display_errors', $oldDisplayErrors);
}
error_reporting($oldErrorReporting);
}
}
/**
* This should generate an Exception "DOMElement::setAttribute(): ID post-60 already defined".
*/
public function testAppendIdAlreadyHere()
public function testAppendIdAlreadyHere(): void
{
$data = '<!DOCTYPE html>
<html lang="fr">
@@ -423,16 +421,15 @@ class ReadabilityTest extends \PHPUnit\Framework\TestCase
</html>';
$readability = $this->getReadability($data, 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
}
public function testPostFilters()
public function testPostFilters(): void
{
$readability = $this->getReadability('<div>' . str_repeat('<p>This <strong>is</strong> the awesome content :)</p>', 10) . '</div>', 'http://0.0.0.0');
$readability->addPostFilter('!<strong[^>]*>(.*?)</strong>!is', '');
@@ -440,63 +437,157 @@ class ReadabilityTest extends \PHPUnit\Framework\TestCase
$res = $readability->init();
$this->assertTrue($res);
$this->assertContains('This the awesome content :)', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This the awesome content :)', $readability->getContent()->getInnerHtml());
}
public function testPreFilters()
public function testPreFilters(): void
{
$this->markTestSkipped('Won\'t work until loadHtml() is moved in init() instead of __construct()');
$readability = $this->getReadability('<div>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>', 'http://0.0.0.0');
$readability->addPreFilter('!<b[^>]*>(.*?)</b>!is', '');
$res = $readability->init();
$this->assertTrue($res);
$this->assertContains('This the awesome and WONDERFUL content :)', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('This the awesome and WONDERFUL content :)', $readability->getContent()->getInnerHtml());
}
public function testChildNodeGoneNull()
public function testChildNodeGoneNull(): void
{
// from http://www.ayyaantuu.net/ethiopia-targets-opposition-lawmakers/
$html = file_get_contents('tests/fixtures/childNodeGoesNull.html');
$html = (string) file_get_contents('tests/fixtures/childNodeGoesNull.html');
$readability = $this->getReadability($html, 'http://0.0.0.0');
$readability->debug = true;
$readability->convertLinksToFootnotes = true;
$res = $readability->init();
$this->assertTrue($res);
}
public function testKeepFootnotes()
public function testKeepFootnotes(): void
{
// from https://www.schreibdichte.de/blog/feed-aggregator-und-spaeter-lesen-dienst-im-team
$html = file_get_contents('tests/fixtures/keepFootnotes.html');
$html = (string) file_get_contents('tests/fixtures/keepFootnotes.html');
$readability = $this->getReadability($html, 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertContains('<sup id="fnref1:fnfeed_2"><a href="#fn:fnfeed_2" class="footnote-ref">2</a></sup>', $readability->getContent()->getInnerHtml());
$this->assertContains('<a href="#fnref1:fnfeed_2" rev="footnote"', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('<sup id="fnref1:fnfeed_2"><a href="#fn:fnfeed_2" class="footnote-ref">2</a></sup>', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('<a href="#fnref1:fnfeed_2" rev="footnote"', $readability->getContent()->getInnerHtml());
}
public function testWithWipedBody()
public function testWithWipedBody(): void
{
// from https://www.cs.cmu.edu/~rgs/alice-table.html
$html = file_get_contents('tests/fixtures/wipedBody.html');
$html = (string) file_get_contents('tests/fixtures/wipedBody.html');
$readability = $this->getReadability($html, 'http://0.0.0.0', 'libxml', false);
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertContains('<a href="alice-I.html">Down the Rabbit-Hole</a>', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('<a href="alice-I.html">Down the Rabbit-Hole</a>', $readability->getContent()->getInnerHtml());
}
private function getReadability($html, $url = null, $parser = 'libxml', $useTidy = true)
public function dataForVisibleNode(): array
{
return [
'visible node' => [
'<div>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
true,
],
'display=none' => [
'<div style="display:none;">' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
false,
],
'display=inline' => [
'<div style="display:inline;">' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
true,
],
'hidden attribute' => [
'<div hidden>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
false,
],
'missing display' => [
'<div style="color:#ccc;">' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
true,
],
];
}
/**
* @dataProvider dataForVisibleNode
*/
public function testVisibleNode(string $content, bool $shouldBeVisible): void
{
$readability = $this->getReadability($content, 'http://0.0.0.0');
$res = $readability->init();
if ($shouldBeVisible) {
$this->assertStringContainsString('WONDERFUL content', $readability->getContent()->getInnerHtml());
} else {
$this->assertStringNotContainsString('WONDERFUL content', $readability->getContent()->getInnerHtml());
}
}
// https://github.com/wallabag/wallabag/issues/8158
public function testCharsetAfterTitle(): void
{
$readability = $this->getReadability('<!DOCTYPE html><html lang="et"><head><title>Tõde ja õigus I</title> <meta charset="utf-8"></head><body><p>See oli läinud aastasaja kolmanda veerandi lõpul. Päike lähenes silmapiirile, seistes sedavõrd madalas, et enam ei ulatunud valgustama ei mäkke ronivat hobust, kes puutelgedega vankrit vedas, ei vankril istuvat noort naist ega ka ligi kolmekümnelist meest, kes kõndis vankri kõrval.</p></body></html>', 'https://et.wikisource.org/wiki/T%C3%B5de_ja_%C3%B5igus_I/I');
$readability->convertLinksToFootnotes = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertSame('Tõde ja õigus I', $readability->getTitle()->getInnerHtml());
$this->assertStringContainsString('Päike lähenes', $readability->getContent()->getInnerHtml());
}
/**
* @return array<string, array{0: string, 1: string, 2?: bool}>
*/
public function dataForHtmlLang(): array
{
return [
'meta' => [
'<html lang="fr"><head><meta charset="utf-8"></head><body><article>' . str_repeat('<p>Tous les êtres humains naissent libres et égaux en dignité et en droits. Ils sont doués de raison et de conscience et doivent agir les uns envers les autres dans un esprit de fraternité.</p>', 7) . '</article></body></html>',
'fr',
],
'head' => [
'<html lang="fr"><head><title>Foo</title></head><body><article>' . str_repeat('<p>Tous les êtres humains naissent libres et égaux en dignité et en droits. Ils sont doués de raison et de conscience et doivent agir les uns envers les autres dans un esprit de fraternité.</p>', 7) . '</article></body></html>',
'fr',
],
'headless' => [
'<html lang="fr"><body><article>' . str_repeat('<p>Tous les êtres humains naissent libres et égaux en dignité et en droits. Ils sont doués de raison et de conscience et doivent agir les uns envers les autres dans un esprit de fraternité.</p>', 7) . '</article></body></html>',
'fr',
// tidy would add <head> tag.
false,
],
'fragment' => [
'<article>' . str_repeat('<p>Tous les êtres humains naissent libres et égaux en dignité et en droits. Ils sont doués de raison et de conscience et doivent agir les uns envers les autres dans un esprit de fraternité.</p>', 7) . '</article>',
'',
// tidy would add <html>.
false,
],
];
}
/**
* @dataProvider dataForHtmlLang
*/
public function testHtmlLang(string $html, string $lang, bool $useTidy = true): void
{
$readability = $this->getReadability($html, 'http://0.0.0.0', 'libxml', $useTidy);
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf(\DOMDocument::class, $readability->dom);
$this->assertSame($lang, $readability->dom->documentElement->getAttribute('lang'));
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertStringContainsString('êtres', $readability->getContent()->getInnerHtml());
}
private function getReadability(string $html, ?string $url = null, string $parser = 'libxml', bool $useTidy = true): Readability
{
$readability = new Readability($html, $url, $parser, $useTidy);