Compare commits

..
148 Commits
Author SHA1 Message Date
Kevin DecherfandGitHub a8b08d8cb2 Merge pull request #114 from Kdecherf/fix/array-filter
Fix "Undefined array key 0" error
2026-06-21 22:46:10 +02:00
Kevin Decherf 7db754debe Fix "Undefined array key 0" error
If a matching element isn't the first child (e.g. preceded by a whitespace
text node), it may sit at a key other than 0 after some cleanup, leading to
a "Undefined array key 0" error.

This fix is now covered by 6 tests.

Fixes #96

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2026-06-21 17:06:38 +02:00
Jérémy BenoistandGitHub 18bfe842f8 Merge pull request #121 from j0k3r/dependabot/composer/friendsofphp/php-cs-fixer-3.95.8
Update friendsofphp/php-cs-fixer requirement from 3.94.2 to 3.95.8
2026-06-19 10:18:22 +02:00
dependabot[bot]andGitHub 63a0db9109 Update friendsofphp/php-cs-fixer requirement from 3.94.2 to 3.95.8
Updates the requirements on [friendsofphp/php-cs-fixer](https://github.com/PHP-CS-Fixer/PHP-CS-Fixer) to permit the latest version.
- [Release notes](https://github.com/PHP-CS-Fixer/PHP-CS-Fixer/releases)
- [Changelog](https://github.com/PHP-CS-Fixer/PHP-CS-Fixer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/PHP-CS-Fixer/PHP-CS-Fixer/compare/v3.94.2...v3.95.8)

---
updated-dependencies:
- dependency-name: friendsofphp/php-cs-fixer
  dependency-version: 3.95.8
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-19 08:08:54 +00:00
Jan Tojnar 63d6ef1a47 .github: Set up Dependabot
This will make GitHub check dependencies in `composer.json` once a week
and open PRs trying to bump each of them whenever a new version is released.
If the CI fails, as is likely the case with PHP-CS-Fixer, we will need
to fix is manually before the PR can be merged.

https://docs.github.com/en/code-security/tutorials/secure-your-dependencies/dependabot-quickstart
2026-06-19 10:05:13 +02:00
Jan Tojnar 116b6c839a JSLikeHTMLElement: Add types to methods
PHP-CS-Fixer 3.95.0 added `void_return` rule to `@Symfony:risky` ruleset.

https://github.com/PHP-CS-Fixer/PHP-CS-Fixer/releases/tag/v3.95.0
https://cs.symfony.com/doc/rules/function_notation/void_return.html

`void` type declaration is available since PHP 7.1:
https://www.php.net/manual/en/language.types.void.php

Magic methods can have type annotations but they must be compatible with the default ones
(not enforced prior to PHP 8.0):
https://php.watch/versions/8.0/magic-method-signatures

Let’s just type all the methods.

This could technically be a BC break if someone is extending
the `JSLikeHTMLElement` class, overriding the methods that previously
did not have return type hint without adding it, and calling
`registerNodeClass` on the public `dom` property but I think it is
pretty unlikely – at least there seem to be no public instances on GitHub.
2026-06-19 10:00:39 +02:00
Jan Tojnar a35f00ebb3 php-cs-fixer: Prevent removing strict_types declaration
PHP-CS-Fixer 3.95.0 started removing the declarations as a part of
`@Symfony:risky` ruleset.

We currently only use it in `rector.php` but it works there so no point
in following Symfony code style on that.

We will probably want to enable `strict_types` globally eventually,
but let’s at least stop removing them for now so that we can bump
PHP-CS-Fixer.

https://github.com/PHP-CS-Fixer/PHP-CS-Fixer/releases/tag/v3.95.0
https://cs.symfony.com/doc/rules/strict/declare_strict_types.html
2026-06-19 09:51:35 +02:00
Jan Tojnar 0f9f971556 tests: Use class constants instead of strings
Fixed using `composer rector` and then `composer fix`:

https://getrector.com/rule-detail/string-class-name-to-class-constant-rector
2026-06-19 09:50:01 +02:00
Jan Tojnar fd1eaf61a7 composer: Stop using lock file
Since we are a library that supports wide range of PHP versions,
we cannot lock into a single set of versions of run-time dependencies.

https://getcomposer.org/doc/02-libraries.md#lock-file

Composer 1.10.0 introduced `lock` option to prevent generation of `composer.lock` file:

https://github.com/composer/composer/releases/tag/1.10.0
https://getcomposer.org/doc/06-config.md#lock

This will basically make `composer install` work the same way as `composer update`.
2026-06-19 05:36:17 +02:00
Jan Tojnar 3e40d78a67 Pin development dependencies
PHP-CS-Fixer, PHPStan and Rector regularly introduce changes which break CI
in minor versions, forcing us to mix unrelated changes with CI fixes.

Let’s pin the minor version and update it on our own time e.g. with Dependabot.

This is also what PHPStan suggests for projects not using `composer.lock`:
https://phpstan.org/user-guide/backward-compatibility-promise#minimizing-impact-of-type-inference-changes
2026-06-19 05:17:48 +02:00
Jérémy BenoistandGitHub 03533f5e4f Merge pull request #111 from j0k3r/fix/tidy-php85
Fix bad encoding for `tidy_repair_string`
2026-03-27 20:34:19 +01:00
Jeremy Benoist 009b4ab9b8 Add PHP 8.5 on CI 2026-03-27 20:31:52 +01:00
Jeremy Benoist f9e73fb49a Fix bad encoding for tidy_repair_string
Tidy on PHP 8.5 is more restrictive on what can be given as input encoding. Before, it worked as an unknown value was converted to `utf8`.
2026-03-27 20:29:09 +01:00
Jérémy BenoistandGitHub 3042990efc Merge pull request #106 from jtojnar/encode
Fix character decoding regression when `title` precedes `meta[charset]`
2025-06-03 09:22:21 +02:00
Jan Tojnar 8b89d70b1a Fix character decoding regression when title precedes meta[charset]
Because of PHP 8.2 deprecation, in f14428e4c0, we stopped converting non-ASCII characters to HTML entities. Instead, we started to explicitly insert `meta[charset]` tag at the start of the document.

Later, we discovered that was breaking `html[lang]` so, in efbbc86df9, we made the insertion smarter. One of the improvements was that it would not insert the `meta[charset]` tag when it was already present.

That, however, broke websites that had `title` tag before `meta[charset]`. On those, libxml2 would decode the `title` contents as ISO-8859-1.

We could improve the logic (e.g. check that there is not text content before `meta[charset]`) or insert the tag unconditionally but it will probably be simplest to just go back to converting the non-ASCII characters to entities, just using non-deprecated function variant.
2025-05-28 01:10:59 +02:00
Jan Tojnar 3e9b15db46 tests: Check encoding was preserved in testHtmlLang
The fix introduced in efbbc86df9 alongside this test also manipulates `meta[charset]` but we were not checking if it does not break encoding.
2025-05-28 00:51:49 +02:00
Jérémy BenoistandGitHub 7413a38ff0 Merge pull request #104 from jtojnar/html-shadowing
Fix discarding `html[lang]`
2025-03-04 10:20:28 +01:00
Jérémy BenoistandGitHub a18cd0f2a9 Merge pull request #102 from jtojnar/local-no-domain
Do not set domainRegExp for local files
2025-03-04 10:20:15 +01:00
Jan Tojnar efbbc86df9 Fix discarding html[lang]
`DOMDocument::loadHTML` will parse HTML documents as ISO-8859-1 if there is no `meta[charset]` tag. This means that UTF-8-encoded HTML fragments such as those coming from JSON-LD `articleBody` field would be parsed with incorrect encoding.

In f14428e4c0, we tried to resolve it by putting `meta[charset]` tag at the start of the HTML fragment. Unfortunately, it turns out that causes parser to auto-insert a `html` element, losing the attributes of the original `html` tag.

Let’s try to insert the `meta[charset]` tag into the proper place in the HTML document.

We do not need to use the same trick with `JSLikeHTMLElement::__set`.
That expects smaller HTML fragments, not `html` documents, so creating `html` and `head` elements will not be a problem.
2025-03-04 01:51:34 +01:00
Jan Tojnar 541fab34a0 tests: Remove pointless debug assignment
It is unused since 8ab7d76cd5.
2025-03-03 23:55:32 +01:00
Jan Tojnar 90869d877e tests: Use ::class for DOMDocument class name
Also capitalize it properly.
2025-03-03 23:53:33 +01:00
Jan Tojnar c7208f6ad2 Do not set domainRegExp for local files
`parse_url($this->url, \PHP_URL_HOST)` will return `null` for local filesystem path.
Casting it to `string` will produce an empty regular expression,
which would match any link when computing link density.
2025-03-03 23:26:30 +01:00
Jérémy BenoistandGitHub 4258559b8a Merge pull request #100 from jtojnar/phpunit-bridge7
composer: Allow phpunit-bridge 7.0
2025-02-24 09:50:46 +01:00
Jan Tojnar 1ac761d708 composer: Allow phpunit-bridge 7.0 2025-02-24 09:47:14 +01:00
Jérémy BenoistandGitHub d3053fbce4 Merge pull request #99 from jtojnar/phpstan2
phpstan: Upgrade to version 2
2025-02-24 07:37:59 +01:00
Jan Tojnar 4c929754e9 phpstan: Upgrade to version 2
https://github.com/phpstan/phpstan/blob/2.1.x/UPGRADING.md

Required also bumping Rector since it uses PHPStan internally.
2025-02-23 02:49:08 +01:00
Jan Tojnar 1d7cdf3a12 phpstan: Use standard config path
This allows developer to create their own own config file, e.g. for setting `editorUrl`:
https://phpstan.org/user-guide/output-format#opening-file-in-an-editor
2025-02-23 02:46:57 +01:00
Jérémy BenoistandGitHub f825dcf55a Merge pull request #90 from jtojnar/foreaches
Iterate node lists with foreach
2024-10-11 08:56:35 +02:00
Jan Tojnar 9a9373de4b Iterate node lists with foreach
`DOMNodeList` implements `Traversable`.

There are some `for` loops left but we cannot simply replace those:
PHP follows the DOM specification, which requires that `NodeList`
objects in the DOM are live. As a result, any operation that removes
a node list member node from its parent (such as `removeChild`,
`replaceChild` or `appendChild`) will cause the next node
in the iterator to be skipped.

We could work around that by converting those node lists to static arrays
using `iterator_to_array` but not sure if it is worth it.
2024-10-10 09:01:45 +02:00
Jan Tojnar d454c3a462 Remove dead iteration code
This was forgotten in b580cf216d.
2024-10-10 09:01:45 +02:00
Jan Tojnar 8b1ef07401 Extract for-iterated items into variables
This simplifies the code a bit and will make it slightly easier in case we decide to switch to `foreach` iteration.
2024-10-10 09:01:45 +02:00
Jan Tojnar 5885dbbe78 Remove pointless stdClass
`DOMNode::$childNodes` always contained `DOMNodeList`.
2024-10-10 09:01:45 +02:00
Jérémy BenoistandGitHub 6947999782 Merge pull request #92 from jtojnar/ci-fix
ci: Fix & add PHP 8.4
2024-10-10 08:59:28 +02:00
Jan Tojnar da755013aa Remove extra set_error_handler callback argument
It is unused and would cause an error on PHP ≥ 8.0:
https://www.php.net/manual/en/function.set-error-handler.php#refsect1-function.set-error-handler-parameters

Not sure if the handler is even necessary – it was introduced in 175196d6c2 but I did not manage to reproduce the original error (Entity 'nbsp' not defined). It was probably fixed by f2a43b476c.
2024-10-10 08:52:28 +02:00
Jan Tojnar 5b9551d1e3 ci: Add PHP 8.4
PHP 8.4 is in beta, with final version scheduled for November so it is time to start testing it.
2024-10-10 01:27:59 +02:00
Jan Tojnar c7b10dcc45 Avoid E_STRICT constant
It will be deprecated in PHP 8.4 and it is meaningless nowadays anyway:
https://wiki.php.net/rfc/deprecations_php_8_4#remove_e_strict_error_level_and_deprecate_e_strict_constant

The use of the constant was introduced in 175196d6c2.
2024-10-10 01:27:59 +02:00
Jan Tojnar 80adfe870b Fix coding style
With php-cs-fixer 3.64.0, the `native_function_invocation` rule no longer passed.
2024-10-10 01:01:55 +02:00
Jérémy BenoistandGitHub cb6b6ac577 Merge pull request #88 from jtojnar/has-single-fix 2024-03-19 06:02:54 +01:00
Jan Tojnar 677f3f096e Fix hasSingleTagInsideElement method
It would fail for e.g. `<div> <p>foo</p> </div>`.

mozilla/readability uses children for the tag lookup, which return only elements.
PHP does not have children property so b580cf216d
mistakenly used `childNodes` instead, but that can return any node type.

Let’s filter the children ourselves.

Also add comments from mozilla/readability’s `_hasSingleTagInsideElement`.
2024-03-18 23:01:43 +01:00
Jérémy BenoistandGitHub 29122763db Merge pull request #89 from jtojnar/php74
Require PHP 7.4
2024-03-18 09:18:11 +01:00
Jan TojnarandJérémy Benoist 89d3b74259 Rectorize to PHP 7.4
Switches to short anonymous function syntax.
2024-03-18 09:16:43 +01:00
Jan TojnarandJérémy Benoist e792644fe8 Drop PHP < 7.4 support
This will allow us to use flexible heredocs in test,
as well as typed properties and other goodies.

https://www.php.net/releases/7_3_0.php
https://www.php.net/releases/7_4_0.php
2024-03-18 09:16:43 +01:00
Jan TojnarandJérémy Benoist 648d8c605b Update coding style for upcoming PHP-CS-Fixer changes
Once we bump minimum PHP version, we will get newer PHP-CS-Fixer,
which will try to apply this cleanups.

Also manually tweak anonymous functions so that they are cleanly formatted
once we switch to `fn` syntax.
2024-03-18 09:16:43 +01:00
Jérémy BenoistandGitHub f28191a728 Merge pull request #86 from jtojnar/ci-bump
ci: Update actions
2024-03-18 09:12:18 +01:00
Jan Tojnar 2103853a1b ci: Bump coveralls to 2.7.0
- Fixes PHP 8 support https://github.com/php-coveralls/php-coveralls/releases/tag/v2.4.3
2024-03-16 22:30:49 +01:00
Jan Tojnar 7f4c6cfcbd ci: Update actions
Mostly just of nodejs bump:

- https://github.com/actions/checkout/releases/tag/v4.0.0
- https://github.com/ramsey/composer-install/releases/tag/3.0.0
2024-03-16 16:01:16 +01:00
Jérémy BenoistandGitHub 38870cdff1 Merge pull request #80 from jtojnar/stricter
Fix some CI issues
2023-04-03 14:47:32 +02:00
Jan Tojnar 9bdd3b6b2e ci: Add PHP 8.2 and 8.3 2023-03-31 05:26:07 +02:00
Jan Tojnar f14428e4c0 Do not use mb_convert_encoding with HTML-ENTITIES as target encoding
This is deprecated since PHP 8.2:

    Deprecated: mb_convert_encoding(): Handling HTML entities via mbstring is deprecated; use htmlspecialchars, htmlentities, or mb_encode_numericentity/mb_decode_numericentity instead

It was used because `DOMDocument`, which uses libxml2 internally, will parse the HTML as ISO-8859-1, unless the document contains an XML encoding declaration or HTML meta tag setting character set.
Since first such element wins, putting the `meta[charset]` up front will ensure the parser uses the correct encoding, even if the document contains incorrect meta tag (e.g. when the document is converted to UTF-8 without also updating the metadata by the software passing it to Readability).

https://stackoverflow.com/a/39148511/160386
2023-03-31 05:26:07 +02:00
Jan Tojnar 23f824a1ce tests: Fix “THE ERROR HANDLER HAS CHANGED!” 2023-03-31 03:19:22 +02:00
Jan Tojnar 2a57124528 composer: upgrade rector 2023-03-31 03:19:22 +02:00
Jan Tojnar 0975574bdb Rector: Upgrade configuration 2023-03-31 03:19:22 +02:00
Jan Tojnar 9ed89bde92 Fix PHP-Cs-Fixer changes
1) src/Readability.php (braces, no_unneeded_control_parentheses, single_line_comment_spacing, global_namespace_import, no_unused_imports, phpdoc_align)
   2) src/JSLikeHTMLElement.php (phpdoc_separation)

Switch code blocks to Markdown syntax to work around `phpdoc_separation`, ApiGen uses Markdown these days anyway.
2023-03-31 03:14:00 +02:00
Jan Tojnar 2c6c6d5987 PHPStan: Use stable PHPUnit path
phpunit-bridge will create a symlink.
2023-03-31 03:14:00 +02:00
Jan Tojnar c5407ec07c composer: Add scripts for development 2023-03-31 03:14:00 +02:00
Jérémy BenoistandGitHub 7cd8476d38 Merge pull request #79 from j0k3r/fix/psr-log-2-3
Allow `psr/log` 2.0 & 3.0
2022-10-17 22:44:36 +02:00
Jeremy Benoist 82083c872b Allow psr/log 2.0 & 3.0 2022-10-17 22:42:47 +02:00
Kevin DecherfandJeremy Benoist 6689f19956 Strip script and style tags through ::clean() method instead of preg_replace
Huge tags can lead to a failure of preg_replace, thus erasing the whole
fetched content.

Fixes https://github.com/wallabag/wallabag/issues/5847

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-06-13 09:13:23 +02:00
Jérémy BenoistandGitHub 0c0653dad6 Merge pull request #73 from Kdecherf/fix/impr
Fix `isPhrasingContent` conditions, text node replacement
2022-02-16 00:03:37 +01:00
Kevin Decherf 2ab87d7445 Fix isPhrasingContent conditions, text node replacement
It also disables reverting forced paragraph elements as it can break
layouts or corrupt content.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-02-15 20:53:28 +01:00
Jérémy BenoistandGitHub 8af69ad68c Merge pull request #71 from j0k3r/feature/enable-rector
Add Rector
2022-02-04 12:15:57 +01:00
Jeremy Benoist c2a1639b34 Add Rector 2022-02-04 12:13:37 +01:00
Jérémy BenoistandGitHub ccf1b336c5 Merge pull request #64 from Kdecherf/improvements 2022-02-04 05:27:57 +01:00
Kevin Decherf a44c4e5482 Add routine to remove invisible nodes
Readability was previously removing (was trying to actually, see next
section) invisible nodes using a pattern from `unlikelyCandidates`. This
was quite hacky and was removed during a backport of logics from
mozilla/readability. There is still a need to remove them so here we
are. We still use a pattern but specifically against the style
attribute. We also remove nodes with the attribute `hidden`.

The clean feature of tidy actually replaces inline style attributes
with css classes thus preventing readability to detect invisible nodes,
see https://github.com/htacg/tidy-html5/blob/5.6.0/src/clean.c#L1488
We therefore set clean configuration to false.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-02-04 01:23:11 +01:00
Kevin Decherf b580cf216d Backport some logics from mozilla/readability
This change backports several things from mozilla/readability:

- Add child score to all ancestors instead of the first parent only
- Check 5 top candidates and try to find alternative candidates within
  ancestors, this can help to find a better parent and grab more content
- Reduce patterns from `unlikelyCandidates` to the one used by Mozilla
  as ours tend to remove useful nodes
- Score headers (h2 to h6) by default in addition to div, p, td and
  section

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-02-04 01:05:13 +01:00
Jérémy BenoistandGitHub 2e9349f076 Merge pull request #69 from j0k3r/feature/php-7.2
Require PHP >= 7.2
2022-02-02 12:53:28 +01:00
Jeremy Benoist c4bba53dbe Remove Scrutinizer 2022-02-02 12:52:12 +01:00
Jeremy Benoist 66215a6c80 Require PHP >= 7.2
- remove test on Composer v1
- remove deprecated function
- move `loadHtml()` into `init()` instead of `__construct`

Kinda prepare 2.0 version :)
2022-02-02 12:44:24 +01:00
Jérémy BenoistandGitHub b1a20a9575 Merge pull request #68 from open-source-contributions/master
Using assertSame to make assertion equal strict
2021-12-17 12:17:44 +01:00
peter279k 97c02e8ad4 Using assertSame to make assertion equal strict 2021-12-17 19:09:36 +08:00
Jérémy BenoistandGitHub c506b7ebd7 Merge pull request #67 from j0k3r/fix/psr-log-void
Fix deprecated message
2021-11-29 21:06:47 +01:00
Jeremy Benoist d0af21814a Ditch assertContains & assertNotContains 2021-11-29 21:04:56 +01:00
Jeremy Benoist 5b1eba79bd Test on PHP 8 & drop Travis 2021-11-29 21:00:36 +01:00
Jérémy BenoistandGitHub fabf096ce6 Fix deprecated message
> Method "Psr\Log\LoggerAwareInterface::setLogger()" might add "void" as a native return type declaration in the future. Do the same in implementation "Readability\Readability" now to avoid errors or add an explicit @return annotation to suppress this message.
2021-11-29 20:50:10 +01:00
Jérémy BenoistandGitHub 8ce1663238 Merge pull request #66 from Kdecherf/fix/figure 2021-10-29 16:24:41 +02:00
Kevin Decherf eb72a315c4 Clean empty figure tags without ending
See 'Tag omission' https://developer.mozilla.org/en-US/docs/Web/HTML/Element/figure

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2021-10-29 15:36:19 +02:00
Jérémy BenoistandGitHub d5330a9c28 Merge pull request #65 from j0k3r/fix/php-cs
Fix PHPCS config file
2021-10-04 11:42:34 +02:00
Jeremy Benoist 635f8963dc Fix Travis & PHPStan 2021-10-04 11:37:33 +02:00
Jeremy Benoist 19b2a25d96 Fix PHPCS config file 2021-10-04 11:32:34 +02:00
Jérémy BenoistandGitHub 9a490fac07 Merge pull request #52 from nicofrand/master
Skip empty (empty innerHTML) nodes when grabbing article
2021-03-09 11:14:29 +01:00
Jérémy BenoistandGitHub 6f6b1f9e2b Merge pull request #62 from j0k3r/fix/avoid-wiped-body
Body can be wiped without tidy
2021-03-09 10:49:54 +01:00
Jeremy Benoist ea1368fac0 Body can be wiped without tidy
Re-create it in that case.

Also run CS-Fixer.
2021-03-08 11:59:24 +01:00
Jérémy BenoistandGitHub be81eb2a4f Create FUNDING.yml 2020-12-08 09:47:38 +01:00
Jérémy BenoistandGitHub 9632c4df8c Merge pull request #61 from j0k3r/github-actions
Ditch Travis to use GitHub Actions
2020-11-30 14:34:46 +01:00
Jeremy Benoist bd9ca1b2cd Ditch Travis to use GitHub Actions 2020-11-30 14:18:40 +01:00
Jérémy BenoistandGitHub 6c917794a7 Merge pull request #60 from jtojnar/patch-1
readability: stop tidy from wrapping noscript text
2020-11-16 12:02:02 +01:00
Jan TojnarandGitHub 7cea79c23a readability: stop tidy from wrapping noscript text
HTML 4.01 Strict only allows block-level elements within noscript, form and
blockquote. The `enclose-block-text` option fixes the instances when those
elements contain inline elements or text by wrapping the children in paragraphs.

HTML 5 has looser content model and allows noscript elements basically anywhere,
including paragraphs, making the noscript elements inherit the parent element’s
content model. This means that tidy will produce invalid HTML nesting paragraphs
for `p > noscript > text`, a structure that would be invalid on two counts
in HTML 4 Strict profile but is completely valid in HTML 5.

Popular WordPress image lazy-loading code produces precisely that structure
so tidy “corrects” it to invalid code. In a proper HTML parser, the produced
code would force close the outer paragraph, making the noscript element
its sibling instead of a child. The only reason this does not break Graby’s code
for stripping the lazy-loading HTML is that libxml2 contains a bug
counteracting this:

https://gitlab.gnome.org/GNOME/libxml2/-/issues/205

Since all three elements allow flow content in HTML 5, it does not make much
sense to enable this option any more. The only possible issues that could occur
is producing HTML code not conforming to 4.01 Strict but that was never guaranteed,
as our example shows, and having blockquotes contain text nodes not wrapped
in paragraphs, which might be expected by some ancient stylesheets
but that is only minor and easily fixable visual backwards incompatibility.
2020-11-14 22:13:53 +01:00
Jérémy BenoistandGitHub c6425cc28b Merge pull request #58 from j0k3r/fix/test-php8
Enable tests for PHP 8
2020-06-08 09:58:28 +02:00
Jeremy Benoist e6ad806460 Enable tests on PHP 8 2020-06-08 09:24:50 +02:00
Jérémy BenoistandGitHub 3fa88461a0 Merge pull request #57 from j0k3r/fix/html5-parser
Use a new deps for HTML5 parser
2020-06-08 07:59:42 +02:00
Jeremy Benoist 6a8ecf232f Use a new deps for HTML5 parser
`electrolinux/php-html5lib` was quite old and incompatible with the upcoming Composer 2.0.
Jumping to `masterminds/html5` for the same result. Also the lib is maintained.

Also:
- keep README in vendors
- use new Scrutinizer engine
- test with lower deps
- remove php-coveralls dev deps and download the phar during the CI build
2020-06-08 07:04:00 +02:00
Jérémy BenoistandGitHub 52b1ddba57 Merge pull request #55 from j0k3r/fix/cleanup-travis
Cleanup travis
2020-04-12 16:38:09 +02:00
Jeremy Benoist 90c625bb57 Cleanup travis 2020-04-12 16:30:02 +02:00
Jérémy BenoistandGitHub 4d1c3b1777 Merge pull request #54 from j0k3r/fix/phpstan
Fix PHPStan (again)
2020-01-06 14:26:41 +01:00
Jeremy Benoist d649b59414 Fixing PHPUnit versions 2020-01-06 14:22:15 +01:00
Jeremy Benoist 44bebfc3d6 Only install PHPStan when we need it
This is to avoid error when installing it on PHP < 7.1
2020-01-06 14:03:30 +01:00
Jeremy Benoist b1acc9ed73 Fix PHPStan (again)
Also cleanup
2019-11-19 14:09:29 +01:00
Jérémy BenoistandGitHub 9306996b47 Merge pull request #53 from j0k3r/openload.co
Add openload.co to media detection
2019-06-25 17:15:37 +02:00
Jeremy Benoist bb75b4f089 Fix PHPStan 2019-06-25 17:06:49 +02:00
Jeremy Benoist 11d2946904 Add openload.co to media detection 2019-06-25 16:54:38 +02:00
nicofrand ff78c63e6d Skip empty (empty innerHTML) nodes when grabbing article 2019-05-25 16:12:52 +02:00
Jérémy BenoistandGitHub f808c1b0a2 Merge pull request #50 from j0k3r/fix/non-well-formed-numeric-value
Fix “A non well formed numeric value encountered”
2019-05-11 22:07:19 +02:00
Jeremy Benoist bb65caf864 Fix “A non well formed numeric value encountered” 2019-05-11 21:58:11 +02:00
Jérémy BenoistandGitHub de1b1d9775 Merge pull request #48 from Simounet/feature/out-removed-from-negative
\bout removed from negative content
2019-04-23 10:45:13 +02:00
Simounet 2e20f76195 \bout removed from negative content 2019-04-19 12:12:41 +02:00
Jérémy BenoistandGitHub 3c0289bf89 Merge pull request #46 from j0k3r/phpstan
Enable PHPStan
2019-02-07 16:08:33 +01:00
Jeremy Benoist 74d9cc605a Enable PHPStan 2019-02-07 15:51:31 +01:00
Jérémy BenoistandGitHub 6a0f3337a6 Merge pull request #45 from j0k3r/fix/update-fixer-rules
Update fixer rules
2019-02-04 11:29:55 +01:00
Jeremy Benoist 2dce2879bf Update fixer rules
Following graby, wallabag, etc.
2019-02-04 11:21:34 +01:00
Jérémy BenoistandGitHub 49ce4233fa Merge pull request #42 from Kdecherf/fix-tidy
tidy: use tidy_repair_string instead of tidy_parse_string+tidy_clean_repair
2019-02-04 11:17:13 +01:00
Kevin DecherfandJeremy Benoist 26c881d864 tidy: use tidy_repair_string instead of tidy_parse_string+tidy_clean_repair
A change released in tidy 5.6.0 breaks php-tidy when using
tidy_parse_string+tidy_clean_repair and wrap=0, incorrectly wrapping
every single word. Also it seems that $tidy->value should not be used to
retrieve the repaired html as far as it is undocumented and for internal
use.

We replace the call with tidy_repair_string which directly returns the
repaired string.

Relates to https://github.com/htacg/tidy-html5/issues/673
Relates to https://bugs.php.net/bug.php?id=75947

Tests pass.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2019-02-04 11:08:18 +01:00
Jérémy BenoistandGitHub db69fe59a2 Merge pull request #44 from j0k3r/fix/coveralls-upload
Update path to coveralls bin
2019-02-04 11:08:02 +01:00
Jeremy Benoist a78f01f656 Update path to coveralls bin 2019-02-04 11:01:47 +01:00
Jérémy BenoistandGitHub db4508003b Merge pull request #43 from Kdecherf/composer-bump
Composer bump, php-cs bump to v2, travis update
2019-02-04 10:50:17 +01:00
Kevin Decherf 15b12ea2d6 travis: remove PHP < 5.6 and related stuff, phpcs on 7.2, 7.3 not fail
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2019-02-03 16:50:40 +01:00
Kevin Decherf 694f0308fe composer: bump dependencies, move to php-cs v2
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2019-02-03 16:49:21 +01:00
Jérémy BenoistandGitHub bbe9021fe7 Merge pull request #36 from Kdecherf/failing-test
tests: fix possible typo in testPostFilters() leading to failure
2018-12-10 12:02:43 +01:00
Jérémy BenoistandGitHub 9dee4a240d Add some badges 2018-11-28 10:05:23 +01:00
Jérémy BenoistandGitHub eef6d6d456 Merge pull request #41 from j0k3r/testing-7.2-7.3
Updating to 7.2
2018-11-26 16:54:20 +01:00
Jeremy Benoist 30a577c617 Dumping HHVM 2018-11-26 16:42:28 +01:00
Jeremy Benoist 70b393a2a3 Remove custom stuff for 7.2 2018-11-26 16:41:42 +01:00
Jeremy Benoist 9ab6d0d9e8 Updating to 7.2 2018-11-26 16:33:38 +01:00
Jérémy BenoistandGitHub 8712db510e Merge pull request #40 from Simounet/giphy-allowed
Giphy added to allowed medias
2018-11-26 16:31:18 +01:00
Simounet 422c74f29c Giphy added to allowed medias 2018-11-20 19:27:41 +01:00
Jérémy BenoistandGitHub ed3393a79f Merge pull request #38 from Simounet/feature/media-added-to-positive
Media class added to positive candidates
2018-06-05 11:54:53 +00:00
Simounet 63cd304dba Media class added to positive candidates
Fix Mediapart images.
2018-06-05 13:26:39 +02:00
Jérémy BenoistandGitHub 43939f88d9 Merge pull request #37 from Kdecherf/footnotes-candidates
Keep elements with 'footnote' as possible candidates
2017-11-03 18:21:40 +02:00
Kevin Decherf 3a7350a8a7 tests: fix possible typo in testPostFilters() leading to failure
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-11-01 16:46:01 +01:00
Kevin Decherf 4c68cc9f09 Keep elements with 'footnote' as possible candidates
Should fix https://github.com/wallabag/wallabag/issues/3100

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-11-01 16:45:07 +01:00
Jérémy BenoistandGitHub 276684ab6d Merge pull request #35 from j0k3r/travis
Fix PHP 5.3 builds
2017-10-09 17:16:58 +02:00
Jeremy Benoist 163058aeff Fix PHP 5.3 builds 2017-10-09 16:58:21 +02:00
Jérémy BenoistandGitHub 6427dd7371 Merge pull request #32 from j0k3r/html5
Add missing HTML5 class
2017-06-30 16:50:09 +02:00
Jeremy Benoist 613a63c062 CS 2017-06-30 16:42:29 +02:00
Jeremy Benoist 05089bbd03 Add missing HTML5 class 2017-06-30 16:32:37 +02:00
Jérémy BenoistandGitHub 5ac82f3c87 Merge pull request #30 from j0k3r/avoid-php-warning
Avoid php warning
2017-05-19 16:16:20 +02:00
Jeremy Benoist 85fb92a042 Fix tests 2017-05-19 16:02:36 +02:00
Jeremy Benoist f2a43b476c Avoid PHP Warning
This isn't the best solution but the previous one using `@` wasn't really better.
Appending a string into a fragment might generate some warning if the string contains bad entity.
For example `&plus;`.
2017-05-19 15:37:46 +02:00
Jérémy BenoistandGitHub 6def743902 Merge pull request #27 from jtojnar/ext-mbstring
Add mbstring as a dependency
2017-03-18 09:16:30 +01:00
Jan Tojnar 8a44926392 Add mbstring as a dependency 2017-03-18 04:02:50 +01:00
Jérémy BenoistandGitHub 8c7740f073 Merge pull request #26 from j0k3r/fix-phpunit-7.x
Fix PHPUnit error on PHP 7+
2017-03-09 09:41:33 +01:00
Jeremy Benoist 6a4720c951 Fix PHPUnit error on PHP 7+ 2017-03-09 09:21:13 +01:00
Jérémy BenoistandGitHub 7a9a82b543 Update README.md 2017-03-09 09:02:04 +01:00
Jérémy BenoistandGitHub 8a91d36294 Merge pull request #25 from jtojnar/patch-1
Add tidy to suggested dependenies
2017-03-09 09:01:05 +01:00
Jan TojnarandGitHub 669adfb20f Add tidy to suggested dependenies 2017-02-16 15:12:45 +01:00
Jeremy BenoistandGitHub 445b889efb Merge pull request #24 from j0k3r/links
Don't be to hard on 'links' attribute
2017-02-02 16:12:48 +01:00
Jeremy Benoist 8b1c3f147d Don't be to hard on 'links' attribute 2017-02-02 15:57:15 +01:00
Jeremy BenoistandGitHub a505a13d0c Merge pull request #22 from jtojnar/master
Depend on psr/log instead of Monolog
2017-01-14 17:09:58 +01:00
Jan Tojnar 9fe0d07805 Depend on psr/log instead of Monolog 2017-01-14 09:08:15 +01:00
21 changed files with 1754 additions and 1140 deletions
+3 -2
View File
@@ -4,9 +4,10 @@ root = true
; Unix-style newlines ; Unix-style newlines
[*] [*]
end_of_line = LF end_of_line = LF
[*.php]
indent_style = space indent_style = space
indent_size = 4 indent_size = 4
trim_trailing_whitespace = true trim_trailing_whitespace = true
insert_final_newline = true insert_final_newline = true
[.github/**.yml]
indent_size = 2
+3 -1
View File
@@ -4,6 +4,8 @@
/.scrutinizer.yml export-ignore /.scrutinizer.yml export-ignore
/.travis.yml export-ignore /.travis.yml export-ignore
/.php_cs export-ignore /.php_cs export-ignore
/README.md export-ignore
/phpunit.xml.dist export-ignore /phpunit.xml.dist export-ignore
/phpstan.neon export-ignore
/rector.php export-ignore
/.github export-ignore
/tests export-ignore /tests export-ignore
+1
View File
@@ -0,0 +1 @@
github: j0k3r
+6
View File
@@ -0,0 +1,6 @@
version: 2
updates:
- package-ecosystem: "composer"
directory: "/"
schedule:
interval: "weekly"
+47
View File
@@ -0,0 +1,47 @@
name: "CS"
on:
pull_request:
branches:
- master
push:
branches:
- master
jobs:
coding-standards:
name: "CS Fixer & PHPStan"
runs-on: "ubuntu-22.04"
strategy:
matrix:
php:
- "7.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v4"
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
coverage: "none"
php-version: "${{ matrix.php }}"
tools: cs2pr, composer:v2
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v3"
with:
composer-options: "--optimize-autoloader --prefer-dist"
- name: "Run PHP CS Fixer"
run: "php vendor/bin/php-cs-fixer fix --verbose --dry-run --format=checkstyle | cs2pr"
- name: "Install PHPUnit for PHPStan"
run: "php vendor/bin/simple-phpunit install"
- name: "Run PHPStan"
run: "php vendor/bin/phpstan analyse --error-format=checkstyle | cs2pr"
@@ -0,0 +1,151 @@
name: "CI"
on:
pull_request:
branches:
- "master"
push:
branches:
- "master"
env:
fail-fast: true
jobs:
phpunit:
name: "PHPUnit (PHP ${{ matrix.php }})"
runs-on: "ubuntu-22.04"
strategy:
matrix:
php:
- "7.4"
- "8.0"
- "8.1"
- "8.2"
- "8.3"
- "8.4"
- "8.5"
steps:
- name: "Checkout"
uses: "actions/checkout@v4"
with:
fetch-depth: 2
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "none"
tools: composer:v2
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v3"
with:
composer-options: "--optimize-autoloader --prefer-dist"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"
phpunit-coverage:
name: "PHPUnit coverage (PHP ${{ matrix.php }})"
runs-on: "ubuntu-22.04"
strategy:
matrix:
php:
- "8.0"
steps:
- name: "Checkout"
uses: "actions/checkout@v4"
with:
fetch-depth: 2
- name: "Install PHP with Xdebug"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "xdebug"
tools: composer:v2
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v3"
with:
composer-options: "--optimize-autoloader --prefer-dist"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit (with coverage)"
run: "php vendor/bin/simple-phpunit -v --coverage-clover build/logs/clover.xml"
- name: "Retrieve Coveralls phar"
run: "wget https://github.com/php-coveralls/php-coveralls/releases/download/v2.7.0/php-coveralls.phar"
- name: "Enable Coveralls phar"
run: "chmod +x php-coveralls.phar"
- name: "Upload to Coveralls"
run: "php php-coveralls.phar -v -x build/logs/clover.xml"
env:
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
phpunit-lowest:
name: "PHPUnit lowest deps (PHP ${{ matrix.php }})"
runs-on: "ubuntu-22.04"
strategy:
matrix:
php:
- "7.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v4"
with:
fetch-depth: 2
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "none"
tools: composer:v2
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v3"
with:
composer-options: "--optimize-autoloader --prefer-dist"
dependency-versions: "lowest"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"
+3
View File
@@ -2,3 +2,6 @@ vendor/
coverage/ coverage/
composer.lock composer.lock
.php_cs.cache .php_cs.cache
.php-cs-fixer.cache
.phpunit.result.cache
phpstan.neon
+37
View File
@@ -0,0 +1,37 @@
<?php
$finder = (new PhpCsFixer\Finder())
->in(__DIR__)
->exclude(['vendor', 'var', 'web'])
;
return (new PhpCsFixer\Config())
->setUsingCache(true)
->setRiskyAllowed(true)
->setRules([
'@Symfony' => true,
'@Symfony:risky' => true,
'array_syntax' => ['syntax' => 'short'],
'combine_consecutive_unsets' => true,
'heredoc_to_nowdoc' => true,
'no_extra_blank_lines' => ['tokens' => ['break', 'continue', 'extra', 'return', 'throw', 'use', 'parenthesis_brace_block', 'square_brace_block', 'curly_brace_block']],
'no_unreachable_default_argument_value' => true,
'no_useless_else' => true,
'no_useless_return' => true,
'ordered_class_elements' => true,
'ordered_imports' => true,
'php_unit_strict' => false,
'phpdoc_order' => true,
// 'psr4' => true,
'strict_comparison' => true,
'strict_param' => true,
'concat_space' => ['spacing' => 'one'],
// Pulled in by @Symfony:risky but we still support PHP 7.4
'modernize_strpos' => false,
// Pulled in by @Symfony, we cannot add property types until we bump PHP to ≥ 7.4
'no_null_property_initialization' => false,
// @Symfony:risky started removing them but we will probably want to go the opposite direction.
'declare_strict_types' => false,
])
->setFinder($finder)
;
-20
View File
@@ -1,20 +0,0 @@
<?php
return Symfony\CS\Config\Config::create()
->setUsingCache(true)
->level(Symfony\CS\FixerInterface::SYMFONY_LEVEL)
// use default SYMFONY_LEVEL and extra fixers:
->fixers(array(
'concat_with_spaces',
'ordered_use',
'phpdoc_order',
'strict',
'strict_param',
'long_array_syntax',
))
->finder(
Symfony\CS\Finder\DefaultFinder::create()
->in(__DIR__)
->exclude(array('vendor'))
)
;
-2
View File
@@ -1,2 +0,0 @@
tools:
external_code_coverage: false
-53
View File
@@ -1,53 +0,0 @@
language: php
php:
- 5.3.3
- 5.3
- 5.4
- 5.5
- 5.6
- 7.0
- 7.1
- nightly
- hhvm
matrix:
include:
- php: 7.0
env: CS_FIXER=run
fast_finish: true
allow_failures:
- php: hhvm
- php: nightly
# faster builds on new travis setup not using sudo
sudo: false
# cache vendor dirs
cache:
directories:
- vendor
- $HOME/.composer/cache
before_install:
- if [ -n "$GH_TOKEN" ]; then composer config github-oauth.github.com ${GH_TOKEN}; fi;
# disable TLS for composer because openssl is disabled for PHP 5.3.3 on travis
# see: https://blog.travis-ci.com/upcoming_ubuntu_11_10_migration/
- if [[ $TRAVIS_PHP_VERSION = 5.3.3 ]]; then composer config -g -- disable-tls true; fi;
- if [[ $TRAVIS_PHP_VERSION = 5.3.3 ]]; then composer config -g -- secure-http false; fi;
# no version compatible with PHP > 7.1
- if [[ $TRAVIS_PHP_VERSION = 7.2.* ]]; then composer remove friendsofphp/php-cs-fixer; fi;
install:
- composer self-update
before_script:
- composer install --prefer-dist --no-interaction
script:
- mkdir -p build/logs
- phpunit -v --coverage-clover build/logs/clover.xml
- if [ "$CS_FIXER" = "run" ]; then php vendor/bin/php-cs-fixer fix --verbose --dry-run ; fi;
after_script:
- php vendor/bin/coveralls -v
+6 -4
View File
@@ -1,9 +1,11 @@
# Readability # Readability
[![Build Status](https://travis-ci.org/j0k3r/php-readability.svg?branch=master)](https://travis-ci.org/j0k3r/php-readability) ![CI](https://github.com/j0k3r/php-readability/workflows/CI/badge.svg)
[![Coverage Status](https://coveralls.io/repos/j0k3r/php-readability/badge.svg?branch=master&service=github)](https://coveralls.io/github/j0k3r/php-readability/?branch=master) [![Coverage Status](https://coveralls.io/repos/j0k3r/php-readability/badge.svg?branch=master&service=github)](https://coveralls.io/github/j0k3r/php-readability/?branch=master)
[![Total Downloads](https://poser.pugx.org/j0k3r/php-readability/downloads)](https://packagist.org/packages/j0k3r/php-readability)
[![License](https://poser.pugx.org/j0k3r/php-readability/license)](https://packagist.org/packages/j0k3r/php-readability)
This is an extract of the Readability class from this [full-text-rss](https://github.com/Dither/full-text-rss) fork. It can be defined as a better version of the original [php-readability](https://bitbucket.org/fivefilters/php-readability/overview). This is an extract of the Readability class from this [full-text-rss](https://github.com/Dither/full-text-rss) fork. It can be defined as a better version of the original [php-readability](https://bitbucket.org/fivefilters/php-readability).
## Differences ## Differences
@@ -17,9 +19,9 @@ The default php-readability lib is really old and needs to be improved. I found
## Requirements ## Requirements
By default, this lib will use the [Tidy extension](https://github.com/htacg/tidy-html5) if it's available. Tidy is only used to cleanup the given HTML and avoid problems with bad HTML structure, etc .. By default, this lib will use the [Tidy extension](https://github.com/htacg/tidy-html5) if it's available. Tidy is only used to cleanup the given HTML and avoid problems with bad HTML structure, etc .. It'll be suggested by Composer.
Since Composer doesn't support suggestion on PHP extension, I write this suggestion here. Also, if you got problem from parsing a content without Tidy installed, please install it and try again.
## Usage ## Usage
+25 -4
View File
@@ -24,14 +24,35 @@
"role": "Developer (original JS version)" "role": "Developer (original JS version)"
}], }],
"require": { "require": {
"php": ">=5.3.3", "php": ">=7.4.0",
"monolog/monolog": "^1.13" "ext-mbstring": "*",
"psr/log": "^1.0.1 || ^2.0 || ^3.0",
"masterminds/html5": "^2.7"
}, },
"require-dev": { "require-dev": {
"satooshi/php-coveralls": "~0.6", "friendsofphp/php-cs-fixer": "3.95.8",
"friendsofphp/php-cs-fixer": "<2" "monolog/monolog": "^1.24|^2.1",
"symfony/phpunit-bridge": "^4.4|^5.3|^6.0|^7.0",
"phpstan/phpstan": "2.2.2",
"phpstan/phpstan-phpunit": "2.0.16",
"rector/rector": "2.4.6"
},
"suggest": {
"ext-tidy": "Used to clean up given HTML and to avoid problems with bad HTML structure."
}, },
"autoload": { "autoload": {
"psr-4": { "Readability\\": "src/" } "psr-4": { "Readability\\": "src/" }
},
"autoload-dev": {
"psr-4": { "Tests\\Readability\\": "tests/" }
},
"config": {
"lock": false
},
"scripts": {
"fix": "php-cs-fixer fix --verbose --diff",
"phpstan": "phpstan analyze --memory-limit 512M",
"rector": "rector process",
"test": "simple-phpunit -v"
} }
} }
+13
View File
@@ -0,0 +1,13 @@
parameters:
level: 1
paths:
- src
- tests
# https://github.com/phpstan/phpstan/issues/694#issuecomment-350724288
bootstrapFiles:
- vendor/bin/.phpunit/phpunit/vendor/autoload.php
includes:
- vendor/phpstan/phpstan-phpunit/extension.neon
- vendor/phpstan/phpstan-phpunit/rules.neon
+2 -3
View File
@@ -7,12 +7,11 @@
convertWarningsToExceptions="true" convertWarningsToExceptions="true"
processIsolation="false" processIsolation="false"
stopOnFailure="false" stopOnFailure="false"
syntaxCheck="false"
bootstrap="vendor/autoload.php" bootstrap="vendor/autoload.php"
> >
<testsuites> <testsuites>
<testsuite name="Readability Test Suite"> <testsuite name="Readability">
<directory>./tests/</directory> <directory>./tests/</directory>
</testsuite> </testsuite>
</testsuites> </testsuites>
@@ -27,6 +26,6 @@
</filter> </filter>
<!-- <logging> <!-- <logging>
<log type="coverage-html" target="coverage" title="Readability" charset="UTF-8" yui="true" highlight="true" lowUpperBound="35" highLowerBound="70"/> <log type="coverage-html" target="coverage" lowUpperBound="35" highLowerBound="70"/>
</logging> --> </logging> -->
</phpunit> </phpunit>
+18
View File
@@ -0,0 +1,18 @@
<?php
declare(strict_types=1);
use Rector\Config\RectorConfig;
use Rector\Set\ValueObject\LevelSetList;
return RectorConfig::configure()
->withPaths([
__DIR__ . '/src',
__DIR__ . '/tests',
])
->withBootstrapFiles([
__DIR__ . '/vendor/bin/.phpunit/phpunit/vendor/autoload.php',
__DIR__ . '/vendor/autoload.php',
])
->withSets([LevelSetList::UP_TO_PHP_74])
;
+68 -48
View File
@@ -39,87 +39,107 @@ class JSLikeHTMLElement extends \DOMElement
/** /**
* Used for setting innerHTML like it's done in JavaScript:. * Used for setting innerHTML like it's done in JavaScript:.
* *
* @code * ```php
* $div->innerHTML = '<h2>Chapter 2</h2><p>The story begins...</p>'; * $div->innerHTML = '<h2>Chapter 2</h2><p>The story begins...</p>';
* @endcode * ```
*/ */
public function __set($name, $value) public function __set(string $name, string $value): void
{ {
if ($name === 'innerHTML') { if ('innerHTML' !== $name) {
// first, empty the element $trace = debug_backtrace();
trigger_error('Undefined property via __set(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], \E_USER_NOTICE);
return;
}
// first, empty the element
if (isset($this->childNodes)) {
for ($x = $this->childNodes->length - 1; $x >= 0; --$x) { for ($x = $this->childNodes->length - 1; $x >= 0; --$x) {
$this->removeChild($this->childNodes->item($x)); $this->removeChild($this->childNodes->item($x));
} }
}
// $value holds our new inner HTML // $value holds our new inner HTML
if ($value !== '') { $value = trim($value);
$f = $this->ownerDocument->createDocumentFragment(); if (empty($value)) {
return;
}
// appendXML() expects well-formed markup (XHTML) // ensure bad entity won't generate warning
// @ to suppress PHP warnings $previousError = libxml_use_internal_errors(true);
$result = @$f->appendXML($value);
if ($result) {
if ($f->hasChildNodes()) {
$this->appendChild($f);
}
} else {
// $value is probably ill-formed
$f = new \DOMDocument();
$value = mb_convert_encoding($value, 'HTML-ENTITIES', 'UTF-8');
// Using <htmlfragment> will generate a warning, but so will bad HTML $f = $this->ownerDocument->createDocumentFragment();
// (and by this point, bad HTML is what we've got).
// We use it (and suppress the warning) because an HTML fragment will
// be wrapped around <html><body> tags which we don't really want to keep.
// Note: despite the warning, if loadHTML succeeds it will return true.
$result = @$f->loadHTML('<htmlfragment>' . $value . '</htmlfragment>');
if ($result) { // appendXML() expects well-formed markup (XHTML)
$import = $f->getElementsByTagName('htmlfragment')->item(0); $result = $f->appendXML($value);
if ($result) {
foreach ($import->childNodes as $child) { if ($f->hasChildNodes()) {
$importedNode = $this->ownerDocument->importNode($child, true); $this->appendChild($f);
$this->appendChild($importedNode);
}
} else {
// oh well, we tried, we really did. :(
// this element is now empty
}
}
} }
} else { } else {
$trace = debug_backtrace(); // $value is probably ill-formed
trigger_error('Undefined property via __set(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], E_USER_NOTICE); $f = new \DOMDocument();
// Using <htmlfragment> will generate a warning, but so will bad HTML
// (and by this point, bad HTML is what we've got).
// We use it (and suppress the warning) because an HTML fragment will
// be wrapped around <html><body> tags which we don't really want to keep.
// Note: despite the warning, if loadHTML succeeds it will return true.
$result = $f->loadHTML('<meta charset="utf-8"><htmlfragment>' . $value . '</htmlfragment>');
if ($result) {
$import = $f->getElementsByTagName('htmlfragment')->item(0);
foreach ($import->childNodes as $child) {
$importedNode = $this->ownerDocument->importNode($child, true);
$this->appendChild($importedNode);
}
}
} }
libxml_clear_errors();
libxml_use_internal_errors($previousError);
} }
/** /**
* Used for getting innerHTML like it's done in JavaScript:. * Used for getting innerHTML like it's done in JavaScript:.
* *
* @code * ```php
* $string = $div->innerHTML; * $string = $div->innerHTML;
* @endcode * ```
*/ */
public function __get($name) public function __get(string $name): string
{ {
if ($name === 'innerHTML') { if ('innerHTML' === $name) {
$inner = ''; $inner = '';
foreach ($this->childNodes as $child) { if (isset($this->childNodes)) {
$inner .= $this->ownerDocument->saveXML($child); foreach ($this->childNodes as $child) {
$inner .= $this->ownerDocument->saveXML($child);
}
} }
return $inner; return $inner;
} }
$trace = debug_backtrace(); $trace = debug_backtrace();
trigger_error('Undefined property via __get(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], E_USER_NOTICE); trigger_error('Undefined property via __get(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], \E_USER_NOTICE);
return; return '';
} }
public function __toString() public function __toString(): string
{ {
return '[' . $this->tagName . ']'; return '[' . $this->tagName . ']';
} }
public function getInnerHtml(): string
{
return $this->__get('innerHTML');
}
public function setInnerHtml(string $value): void
{
$this->__set('innerHTML', $value);
}
} }
+872 -789
View File
File diff suppressed because it is too large Load Diff
+378 -214
View File
@@ -4,388 +4,378 @@ namespace Tests\Readability;
use Monolog\Handler\TestHandler; use Monolog\Handler\TestHandler;
use Monolog\Logger; use Monolog\Logger;
use Psr\Log\LoggerInterface;
use Readability\JSLikeHTMLElement;
use Readability\Readability; use Readability\Readability;
class ReadabilityTest extends \PHPUnit_Framework_TestCase class ReadabilityTest extends \PHPUnit\Framework\TestCase
{ {
/** @var TestHandler */
public $logHandler; public $logHandler;
/** @var LoggerInterface */
public $logger; public $logger;
private function getReadability($html, $url = null, $parser = 'libxml', $useTidy = true)
{
$readability = new Readability($html, $url, $parser, $useTidy);
$this->logHandler = new TestHandler();
$this->logger = new Logger('test', array($this->logHandler));
$readability->setLogger($this->logger);
return $readability;
}
/** /**
* @requires extension tidy * @requires extension tidy
*/ */
public function testConstructDefault() public function testConstructDefault(): void
{ {
$readability = $this->getReadability(''); $readability = $this->getReadability('');
$this->assertSame('utf8', $readability->tidy_config['input-encoding']);
$readability->init();
$this->assertNull($readability->url); $this->assertNull($readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom); $this->assertInstanceOf(\DOMDocument::class, $readability->dom);
}
public function testConstructHtml5Parser(): void
{
$readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'html5lib');
$readability->init();
$this->assertSame('http://0.0.0.0', $readability->url);
$this->assertInstanceOf(\DOMDocument::class, $readability->dom);
$this->assertSame('<html/>', $readability->original_html);
} }
/** /**
* @requires extension tidy * @requires extension tidy
*/ */
public function testConstructSimple() public function testConstructSimple(): void
{ {
$readability = $this->getReadability('<html/>', 'http://0.0.0.0'); $readability = $this->getReadability('<html/>', 'http://0.0.0.0');
$readability->init();
$this->assertEquals('http://0.0.0.0', $readability->url); $this->assertSame('http://0.0.0.0', $readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom); $this->assertInstanceOf(\DOMDocument::class, $readability->dom);
$this->assertEquals('<html/>', $readability->original_html); $this->assertSame('<html/>', $readability->original_html);
$this->assertTrue($readability->tidied); $this->assertTrue($readability->tidied);
} }
public function testConstructDefaultWithoutTidy() public function testConstructDefaultWithoutTidy(): void
{ {
$readability = $this->getReadability('', null, 'libxml', false); $readability = $this->getReadability('', null, 'libxml', false);
$readability->init();
$this->assertNull($readability->url); $this->assertNull($readability->url);
$this->assertEquals('', $readability->original_html); $this->assertSame('', $readability->original_html);
$this->assertFalse($readability->tidied); $this->assertFalse($readability->tidied);
$this->assertInstanceOf('DomDocument', $readability->dom); $this->assertInstanceOf(\DOMDocument::class, $readability->dom);
} }
public function testConstructSimpleWithoutTidy() public function testConstructSimpleWithoutTidy(): void
{ {
$readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'libxml', false); $readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'libxml', false);
$readability->init();
$this->assertEquals('http://0.0.0.0', $readability->url); $this->assertSame('http://0.0.0.0', $readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom); $this->assertInstanceOf(\DOMDocument::class, $readability->dom);
$this->assertEquals('<html/>', $readability->original_html); $this->assertSame('<html/>', $readability->original_html);
$this->assertFalse($readability->tidied); $this->assertFalse($readability->tidied);
} }
public function testInitNoContent() public function testInitNoContent(): void
{ {
$readability = $this->getReadability('<html/>', 'http://0.0.0.0'); $readability = $this->getReadability('<html/>', 'http://0.0.0.0');
$res = $readability->init(); $res = $readability->init();
$this->assertFalse($res); $this->assertFalse($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('Sorry, Readability was unable to parse this page for content.', $readability->getContent()->innerHTML); $this->assertStringContainsString('Sorry, Readability was unable to parse this page for content.', $readability->getContent()->getInnerHtml());
} }
public function testInitP() public function testInitP(): void
{ {
$readability = $this->getReadability(str_repeat('<p>This is the awesome content :)</p>', 7), 'http://0.0.0.0'); $readability = $this->getReadability(str_repeat('<p>This is the awesome content :)</p>', 7), 'http://0.0.0.0');
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is the awesome content :)', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is the awesome content :)', $readability->getContent()->getInnerHtml());
} }
public function testInitDivP() public function testInitDivP(): void
{ {
$readability = $this->getReadability('<div>' . str_repeat('<p>This is the awesome content :)</p>', 7) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('<p>This is the awesome content :)</p>', 7) . '</div>', 'http://0.0.0.0');
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is the awesome content :)', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is the awesome content :)', $readability->getContent()->getInnerHtml());
} }
public function testInitDiv() public function testInitDiv(): void
{ {
$readability = $this->getReadability('<div>' . str_repeat('This is the awesome content :)', 7) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('This is the awesome content :)', 7) . '</div>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is the awesome content :)', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is the awesome content :)', $readability->getContent()->getInnerHtml());
} }
public function testWithFootnotes() public function testWithFootnotes(): void
{ {
$readability = $this->getReadability('<div>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '</div>', 'http://0.0.0.0');
$readability->debug = true;
$readability->convertLinksToFootnotes = true; $readability->convertLinksToFootnotes = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('readabilityFootnoteLink', $readability->getContent()->innerHTML); $this->assertStringContainsString('readabilityFootnoteLink', $readability->getContent()->getInnerHtml());
$this->assertContains('readabilityLink-3', $readability->getContent()->innerHTML); $this->assertStringContainsString('readabilityLink-3', $readability->getContent()->getInnerHtml());
} }
public function testStandardClean() public function testStandardClean(): void
{ {
$readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<a href="#nofollow" rel="nofollow">will NOT be removed</a></div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<a href="#nofollow" rel="nofollow">will NOT be removed</a></div>', 'http://0.0.0.0');
$readability->debug = true;
$readability->lightClean = false; $readability->lightClean = false;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('will NOT be removed', $readability->getContent()->innerHTML); $this->assertStringContainsString('will NOT be removed', $readability->getContent()->getInnerHtml());
$this->assertNotContains('<h2>', $readability->getContent()->innerHTML); $this->assertStringNotContainsString('<h2>', $readability->getContent()->getInnerHtml());
} }
public function testWithIframe() public function testWithIframe(): void
{ {
$readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe><iframe>http://soundcloud.com/test</iframe></p></div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe><iframe>http://soundcloud.com/test</iframe></p></div>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('nofollow', $readability->getContent()->innerHTML); $this->assertStringContainsString('nofollow', $readability->getContent()->getInnerHtml());
} }
public function testWithArticle() public function testWithArticle(): void
{ {
$readability = $this->getReadability('<article><p>' . str_repeat('This is an awesome text with some links, here there are: the awesome', 20) . '</p><p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article><p>' . str_repeat('This is an awesome text with some links, here there are: the awesome', 20) . '</p><p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->innerHTML); $this->assertStringContainsString('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('nofollow', $readability->getContent()->innerHTML); $this->assertStringContainsString('nofollow', $readability->getContent()->getInnerHtml());
} }
public function testWithAside() public function testWithAside(): void
{ {
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<footer><aside>' . str_repeat('<p>This is an awesome text with some links, here there are</p>', 8) . '</aside></footer></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<footer><aside>' . str_repeat('<p>This is an awesome text with some links, here there are</p>', 8) . '</aside></footer></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('<aside>', $readability->getContent()->innerHTML); $this->assertStringNotContainsString('<aside>', $readability->getContent()->getInnerHtml());
$this->assertContains('<footer readability="4"/>', $readability->getContent()->innerHTML); $this->assertStringContainsString('<footer readability="9"/>', $readability->getContent()->getInnerHtml());
} }
public function testWithClasses() public function testWithClasses(): void
{ {
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->innerHTML); $this->assertStringContainsString('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text should be removed', $readability->getContent()->innerHTML); $this->assertStringNotContainsString('This text should be removed', $readability->getContent()->getInnerHtml());
} }
public function testWithClassesWithoutLightClean() public function testWithClassesWithoutLightClean(): void
{ {
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0');
$readability->debug = true;
$readability->lightClean = false; $readability->lightClean = false;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->innerHTML); $this->assertStringContainsString('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text should be removed', $readability->getContent()->innerHTML); $this->assertStringNotContainsString('This text should be removed', $readability->getContent()->getInnerHtml());
} }
public function testWithTd() public function testWithTd(): void
{ {
$readability = $this->getReadability('<table><tr>' . str_repeat('<td><p>This is an awesome text with some links, here there are the awesome</td>', 7) . '</tr></table>', 'http://0.0.0.0'); $readability = $this->getReadability('<table><tr>' . str_repeat('<td><p>This is an awesome text with some links, here there are the awesome</td>', 7) . '</tr></table>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
} }
public function testWithSameClasses() public function testWithSameClasses(): void
{ {
$readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<div class="awesomecontent">This text is also an awesome text and you should know that !</div></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<div class="awesomecontent">This text is also an awesome text and you should know that !</div></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML); $this->assertStringContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testWithScript() public function testWithScript(): void
{ {
$readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p><script>This text is also an awesome text and you should know that !</script></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p><script>This text is also an awesome text and you should know that !</script></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testTitle() public function testTitle(): void
{ {
$readability = $this->getReadability('<title>this is my title</title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<title>this is my title</title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEquals('this is my title', $readability->getTitle()->innerHTML); $this->assertSame('this is my title', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testTitleWithDash() public function testTitleWithDash(): void
{ {
$readability = $this->getReadability('<title> title2 - title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<title> title2 - title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEquals('title2 - title3', $readability->getTitle()->innerHTML); $this->assertSame('title2 - title3', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testTitleWithDoubleDot() public function testTitleWithDoubleDot(): void
{ {
$readability = $this->getReadability('<title> title2 : title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<title> title2 : title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEquals('title2 : title3', $readability->getTitle()->innerHTML); $this->assertSame('title2 : title3', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testTitleTooShortUseH1() public function testTitleTooShortUseH1(): void
{ {
$readability = $this->getReadability('<title>too short</title><h1>this is my h1 title !</h1><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<title>too short</title><h1>this is my h1 title !</h1><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEquals('this is my h1 title !', $readability->getTitle()->innerHTML); $this->assertSame('this is my h1 title !', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
// public function testConstructParser() public function testAutoClosingIframeNotThrowingException(): void
// {
// $readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'html5lib');
// $this->assertEquals('http://0.0.0.0', $readability->url);
// }
// dummy function to be used to the next test
public function error2Exception($code, $string, $file, $line, $context)
{ {
throw new \Exception($string, $code); $oldErrorReporting = error_reporting(\E_ALL);
} $oldDisplayErrors = ini_set('display_errors', '1');
// dummy function to be used to the next test
set_error_handler(static function (int $errno, string $errstr, string $errfile, int $errline) {
throw new \Exception($errstr, $errno);
});
public function testAutoClosingIframeNotThrowingException() try {
{ $data = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
error_reporting(E_ALL | E_STRICT); <html xmlns="http://www.w3.org/1999/xhtml" lang="ru-RU" prefix="og: http://ogp.me/ns#">
ini_set('display_errors', true);
set_error_handler(array($this, 'error2Exception'), E_ALL | E_STRICT);
$data = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <head profile="http://gmpg.org/xfn/11">
<html xmlns="http://www.w3.org/1999/xhtml" lang="ru-RU" prefix="og: http://ogp.me/ns#"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<head profile="http://gmpg.org/xfn/11"> </head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <body class="single single-post postid-22030 single-format-standard">
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> <div id="wrapper">
<div id="content">
</head> <div class="post-22030 post type-post status-publish format-standard has-post-thumbnail hentry category-video category-reviews tag-193" id="post-22030">
<body class="single single-post postid-22030 single-format-standard"> <h1>3D Touch &#8212; будущее мобильных игр</h1>
<div id="wrapper"> <div class="postdate">Автор: <strong>Сергей Пак</strong> | Просмотров: 1363 | Опубликовано: 14 сентября 2015 </div>
<div id="content"> <div class="entry">
<div class="post-22030 post type-post status-publish format-standard has-post-thumbnail hentry category-video category-reviews tag-193" id="post-22030"> <p>Компания Apple представила новую технологию 3D Touch, которая является прямым потомком более ранней версии Force Touch &#8212; последняя, напомним, используется сейчас в трекпадах Macbook Pro и Macbook 2015. Теперь управлять устройством стало в разы проще, и Force Touch открывает перед пользователями новые возможности, но при этом 3D Touch &#8212; это про другое. Дело в том, что теперь и на мобильных устройствах интерфейс будет постепенно меняться, кардинальные перемены ждут мобильный гейминг, потому что здесь разработчики действительно могут разгуляться.<span id="more-22030"></span></p>
<h1>3D Touch &#8212; будущее мобильных игр</h1> <p><iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p>
<div class="postdate">Автор: <strong>Сергей Пак</strong> | Просмотров: 1363 | Опубликовано: 14 сентября 2015 </div> <p>Итак, просто представьте себе, что iPhone 6S &#8212; это, по большому счету, отличная игровая приставка, которую вы носите с собой, а еще она может выдавать невероятной красоты картинку. Но проблема заключается, пожалуй, в том, что управлять персонажем в играх довольно трудно &#8212; он неповоротлив, обладает заторможенной реакцией, а игровой клиент зачастую требует перегруза интерфейса для того, чтобы обеспечить максимально большое количество возможностей. Благодаря трехуровневому нажатию можно избавиться от лишних кнопок и обеспечить более качественный обзор местности, и при этом пользователь будет закрывать пальцами минимальное пространство.</p>
<div class="entry"> </div>
<p>Компания Apple представила новую технологию 3D Touch, которая является прямым потомком более ранней версии Force Touch &#8212; последняя, напомним, используется сейчас в трекпадах Macbook Pro и Macbook 2015. Теперь управлять устройством стало в разы проще, и Force Touch открывает перед пользователями новые возможности, но при этом 3D Touch &#8212; это про другое. Дело в том, что теперь и на мобильных устройствах интерфейс будет постепенно меняться, кардинальные перемены ждут мобильный гейминг, потому что здесь разработчики действительно могут разгуляться.<span id="more-22030"></span></p> </div>
<p><iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p>
<p>Итак, просто представьте себе, что iPhone 6S &#8212; это, по большому счету, отличная игровая приставка, которую вы носите с собой, а еще она может выдавать невероятной красоты картинку. Но проблема заключается, пожалуй, в том, что управлять персонажем в играх довольно трудно &#8212; он неповоротлив, обладает заторможенной реакцией, а игровой клиент зачастую требует перегруза интерфейса для того, чтобы обеспечить максимально большое количество возможностей. Благодаря трехуровневому нажатию можно избавиться от лишних кнопок и обеспечить более качественный обзор местности, и при этом пользователь будет закрывать пальцами минимальное пространство.</p>
</div> </div>
</div> </div>
</div> </body>
</div> </html>';
</body>
</html>';
$readability = $this->getReadability($data, 'http://iosgames.ru/?p=22030'); $readability = $this->getReadability($data, 'http://iosgames.ru/?p=22030');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"> </iframe>', $readability->getContent()->innerHTML); $this->assertStringContainsString('<iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"> </iframe>', $readability->getContent()->getInnerHtml());
$this->assertContains('3D Touch', $readability->getTitle()->innerHTML); $this->assertStringContainsString('3D Touch', $readability->getTitle()->getInnerHtml());
} finally {
restore_error_handler();
if (false !== $oldDisplayErrors) {
ini_set('display_errors', $oldDisplayErrors);
}
error_reporting($oldErrorReporting);
}
} }
/** /**
* This should generate an Exception "DOMElement::setAttribute(): ID post-60 already defined". * This should generate an Exception "DOMElement::setAttribute(): ID post-60 already defined".
*/ */
public function testAppendIdAlreadyHere() public function testAppendIdAlreadyHere(): void
{ {
$data = '<!DOCTYPE html> $data = '<!DOCTYPE html>
<html lang="fr"> <html lang="fr">
@@ -432,49 +422,223 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
</html>'; </html>';
$readability = $this->getReadability($data, 'http://0.0.0.0'); $readability = $this->getReadability($data, 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
} }
public function testPostFilters() public function testPostFilters(): void
{ {
$readability = $this->getReadability('<div>' . str_repeat('<p>This <b>is</b> the awesome content :)</p>', 7) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('<p>This <strong>is</strong> the awesome content :)</p>', 10) . '</div>', 'http://0.0.0.0');
$readability->addPostFilter('!<strong[^>]*>(.*?)</strong>!is', ''); $readability->addPostFilter('!<strong[^>]*>(.*?)</strong>!is', '');
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertContains('This the awesome content :)', $readability->getContent()->innerHTML); $this->assertStringContainsString('This the awesome content :)', $readability->getContent()->getInnerHtml());
} }
public function testPreFilters() public function testPreFilters(): void
{ {
$this->markTestSkipped('Won\'t work until loadHtml() is moved in init() instead of __construct()');
$readability = $this->getReadability('<div>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>', 'http://0.0.0.0');
$readability->addPreFilter('!<b[^>]*>(.*?)</b>!is', ''); $readability->addPreFilter('!<b[^>]*>(.*?)</b>!is', '');
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertContains('This the awesome and WONDERFUL content :)', $readability->getContent()->innerHTML); $this->assertStringContainsString('This the awesome and WONDERFUL content :)', $readability->getContent()->getInnerHtml());
} }
public function testChildNodeGoneNull() public function testChildNodeGoneNull(): void
{ {
// from http://www.ayyaantuu.net/ethiopia-targets-opposition-lawmakers/ // from http://www.ayyaantuu.net/ethiopia-targets-opposition-lawmakers/
$html = file_get_contents('tests/fixtures/childNodeGoesNull.html'); $html = (string) file_get_contents('tests/fixtures/childNodeGoesNull.html');
$readability = $this->getReadability($html, 'http://0.0.0.0'); $readability = $this->getReadability($html, 'http://0.0.0.0');
$readability->debug = true;
$readability->convertLinksToFootnotes = true; $readability->convertLinksToFootnotes = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
} }
public function dataForHasSingleTagInsideElement(): array
{
return [
'single matching tag, no other content' => [
'<div><p>Some text</p></div>', 'p', true,
],
'single matching tag with comment' => [
'<div><!-- comment --><p>Some text</p></div>', 'p', true,
],
'whitespace-only text around single matching tag' => [
'<div> <p>Some text</p> </div>', 'p', true,
],
'two matching tags' => [
'<div><p>One</p><p>Two</p></div>', 'p', false,
],
'non-whitespace text alongside the single tag' => [
'<div>Some text<p>One</p></div>', 'p', false,
],
'single tag with a different name' => [
'<div><span>One</span></div>', 'p', false,
],
];
}
/**
* @dataProvider dataForHasSingleTagInsideElement
*/
public function testHasSingleTagInsideElement(string $html, string $tag, bool $expected): void
{
$dom = new \DOMDocument();
$dom->loadHTML('<html><body>' . $html . '</body></html>');
$node = $dom->getElementsByTagName('div')->item(0);
$readability = $this->getReadability('', '', 'libxml', false);
$method = new \ReflectionMethod($readability, 'hasSingleTagInsideElement');
if (\PHP_VERSION_ID < 80100) {
$method->setAccessible(true);
}
$this->assertSame($expected, $method->invoke($readability, $node, $tag));
}
public function testKeepFootnotes(): void
{
// from https://www.schreibdichte.de/blog/feed-aggregator-und-spaeter-lesen-dienst-im-team
$html = (string) file_get_contents('tests/fixtures/keepFootnotes.html');
$readability = $this->getReadability($html, 'http://0.0.0.0');
$res = $readability->init();
$this->assertTrue($res);
$this->assertStringContainsString('<sup id="fnref1:fnfeed_2"><a href="#fn:fnfeed_2" class="footnote-ref">2</a></sup>', $readability->getContent()->getInnerHtml());
$this->assertStringContainsString('<a href="#fnref1:fnfeed_2" rev="footnote"', $readability->getContent()->getInnerHtml());
}
public function testWithWipedBody(): void
{
// from https://www.cs.cmu.edu/~rgs/alice-table.html
$html = (string) file_get_contents('tests/fixtures/wipedBody.html');
$readability = $this->getReadability($html, 'http://0.0.0.0', 'libxml', false);
$res = $readability->init();
$this->assertTrue($res);
$this->assertStringContainsString('<a href="alice-I.html">Down the Rabbit-Hole</a>', $readability->getContent()->getInnerHtml());
}
public function dataForVisibleNode(): array
{
return [
'visible node' => [
'<div>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
true,
],
'display=none' => [
'<div style="display:none;">' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
false,
],
'display=inline' => [
'<div style="display:inline;">' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
true,
],
'hidden attribute' => [
'<div hidden>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
false,
],
'missing display' => [
'<div style="color:#ccc;">' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
true,
],
];
}
/**
* @dataProvider dataForVisibleNode
*/
public function testVisibleNode(string $content, bool $shouldBeVisible): void
{
$readability = $this->getReadability($content, 'http://0.0.0.0');
$res = $readability->init();
if ($shouldBeVisible) {
$this->assertStringContainsString('WONDERFUL content', $readability->getContent()->getInnerHtml());
} else {
$this->assertStringNotContainsString('WONDERFUL content', $readability->getContent()->getInnerHtml());
}
}
// https://github.com/wallabag/wallabag/issues/8158
public function testCharsetAfterTitle(): void
{
$readability = $this->getReadability('<!DOCTYPE html><html lang="et"><head><title>Tõde ja õigus I</title> <meta charset="utf-8"></head><body><p>See oli läinud aastasaja kolmanda veerandi lõpul. Päike lähenes silmapiirile, seistes sedavõrd madalas, et enam ei ulatunud valgustama ei mäkke ronivat hobust, kes puutelgedega vankrit vedas, ei vankril istuvat noort naist ega ka ligi kolmekümnelist meest, kes kõndis vankri kõrval.</p></body></html>', 'https://et.wikisource.org/wiki/T%C3%B5de_ja_%C3%B5igus_I/I');
$readability->convertLinksToFootnotes = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertSame('Tõde ja õigus I', $readability->getTitle()->getInnerHtml());
$this->assertStringContainsString('Päike lähenes', $readability->getContent()->getInnerHtml());
}
/**
* @return array<string, array{0: string, 1: string, 2?: bool}>
*/
public function dataForHtmlLang(): array
{
return [
'meta' => [
'<html lang="fr"><head><meta charset="utf-8"></head><body><article>' . str_repeat('<p>Tous les êtres humains naissent libres et égaux en dignité et en droits. Ils sont doués de raison et de conscience et doivent agir les uns envers les autres dans un esprit de fraternité.</p>', 7) . '</article></body></html>',
'fr',
],
'head' => [
'<html lang="fr"><head><title>Foo</title></head><body><article>' . str_repeat('<p>Tous les êtres humains naissent libres et égaux en dignité et en droits. Ils sont doués de raison et de conscience et doivent agir les uns envers les autres dans un esprit de fraternité.</p>', 7) . '</article></body></html>',
'fr',
],
'headless' => [
'<html lang="fr"><body><article>' . str_repeat('<p>Tous les êtres humains naissent libres et égaux en dignité et en droits. Ils sont doués de raison et de conscience et doivent agir les uns envers les autres dans un esprit de fraternité.</p>', 7) . '</article></body></html>',
'fr',
// tidy would add <head> tag.
false,
],
'fragment' => [
'<article>' . str_repeat('<p>Tous les êtres humains naissent libres et égaux en dignité et en droits. Ils sont doués de raison et de conscience et doivent agir les uns envers les autres dans un esprit de fraternité.</p>', 7) . '</article>',
'',
// tidy would add <html>.
false,
],
];
}
/**
* @dataProvider dataForHtmlLang
*/
public function testHtmlLang(string $html, string $lang, bool $useTidy = true): void
{
$readability = $this->getReadability($html, 'http://0.0.0.0', 'libxml', $useTidy);
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf(\DOMDocument::class, $readability->dom);
$this->assertSame($lang, $readability->dom->documentElement->getAttribute('lang'));
$this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertStringContainsString('êtres', $readability->getContent()->getInnerHtml());
}
private function getReadability(string $html, ?string $url = null, string $parser = 'libxml', bool $useTidy = true): Readability
{
$readability = new Readability($html, $url, $parser, $useTidy);
$this->logHandler = new TestHandler();
$this->logger = new Logger('test', [$this->logHandler]);
$readability->setLogger($this->logger);
return $readability;
}
} }
+54
View File
@@ -0,0 +1,54 @@
<article>
<h1>Feed-Aggregator und Sp&auml;ter-Lesen-Dienst im Team</h1>
<figure>
<img src="https://www.schreibdichte.de/content/1-blog/20170505-feed-aggregator-und-spaeter-lesen-dienst-im-team/workflow.png" alt="Feed-Aggregator und Sp&auml;ter-Lesen-Dienst im Team" >
</figure>
<time datetime="2017-05-05T18:40:00+00:00" >05.05.2017</time>
<p> <span class="tag">Tags:</span>
<ul class="tag_list">
<li><a href="https://www.schreibdichte.de/blog/tag:Software">Software</a></li>
<li><a href="https://www.schreibdichte.de/blog/tag:Web">Web</a></li>
<li><a href="https://www.schreibdichte.de/blog/tag:Tools">Tools</a></li>
</ul>
<p>
<div class="content">
<p>Dank einer Mitteilung <sup id="fnref1:fnfeed_1"><a href="#fn:fnfeed_1" class="footnote-ref">1</a></sup> im OSBN <sup id="fnref1:fnfeed_2"><a href="#fn:fnfeed_2" class="footnote-ref">2</a></sup>, bin ich auf den Feed-Aggregator Miniflux <sup id="fnref1:fnfeed_3"><a href="#fn:fnfeed_3" class="footnote-ref">3</a></sup> aufmerksam geworden. Ich hatte schon lange nach einer Alternative zu Fever <sup id="fnref1:fnfeed_4"><a href="#fn:fnfeed_4" class="footnote-ref">4</a></sup> gesucht, da der Anschein vorlag, dass die Entwicklung nicht mehr weiter ging. Was sich bestätigte.</p>
<p>Miniflux ist prima, einfach gehalten und erledigt genau dass, was es machen soll. Zudem wird die Fever-API unterstützt, was bedeutet, dass ich weiterhin meine Feed-Reader nutzen kann, was den Dialog mit Miniflux auf dem Server betrifft. Das ist deshalb interessant, um den Lesestatus oder das Favorisieren von Feeds zentral auf dem Server zu berücksichtigen und nicht lokal. So habe ich sowohl auf dem lokalen Rechner, als auch auf allen mobilen Geräten den gleichen Stand und Status meiner abonnierten Feeds.</p>
<p>Für das spätere Lesen von Webseiten, nutze ich seit längerem wallabag <sup id="fnref1:fnfeed_5"><a href="#fn:fnfeed_5" class="footnote-ref">5</a></sup>. Wallabag wird mit Inhalten über den Browser gefüttert. Die Webseiten werden bei der Konservierung von Werbung befreit, damit der reine Text vorliegt. Bilder dagegen nicht und gehen evtl. verloren, wenn es die Quelle mal nicht mehr geben sollte. Ich habe also zwei Orte, an denen Informationen gehalten werden. Miniflux zur Verwaltung von Feed-Nachrichten und Wallabag für Webseiten.</p>
<p>Bei einem Update von Miniflux ist mit aufgefallen, dass es nun eine API für Wallabag gibt. Dabei werden alle gespeicherten Feeds von Miniflux an Wallabag übergeben. Eine prima Angelegenheit, da ich jetzt nur noch einen Topf habe, woraus ich meine gesammelten Informationen aus dem Netz entnehme.</p>
<p>Miniflux als auch wallabag sind selbst gehostete Web-Applikationen, die auch über den Browser erreichbar sind. Native wallabag-Apps für iOS bzw. Android werden ebenfalls angeboten. Wallabag bietet zusätzlich einen Feed an, um die gesammelten Informationen per Feed-Reader zu lesen. Hierbei drängt sich evtl. die Idee auf, diesen Feed wieder in Miniflux zu abonnieren, da dann zentral alles aus verschiedenen Quellen vorliegt. Hier hätten wir allerdings eine gewisse Redundanz.</p>
<p>Ich nutze seit ein paar Monaten Devonthink <sup id="fnref1:fnfeed_6"><a href="#fn:fnfeed_6" class="footnote-ref">6</a></sup> auf dem Mac. Das ist eine lokale Dokumentenverwaltung (DMS), die heute u.a. meine Bücher, Notizen und mein Mail-Archiv verwaltet und ihre Stärke u.a. in der Suche von Informationen hat. Devonthink versteht es auch Feeds zu abonnieren. Für mich war dies der Moment, nun den Feed von wallabag mit all meinen gespeicherten Informationen aus dem Web mit zu integrieren. Ich muss sagen, eine runde Lösung, denn jetzt habe ich für die lokale Suche und das Lesen eine zentrale Stelle.</p>
<p>Die Grafik veranschaulicht das Teamplay zwischen Feed-Aggregator Miniflux und wallabag als Später-Lesen-Dienst, zusammengeführt in Devonthink. Ich kenne leider keine Dokumentenverwaltung für Linux oder Windows, die auch Feeds versteht. Wer etwas kennt, bitte melden. Wer zudem etwas zum Thema hören möchte, für dem ist ggf. die Podcast-Folge &quot;Der eigene Workflow&quot; <sup id="fnref1:fnfeed_7"><a href="#fn:fnfeed_7" class="footnote-ref">7</a></sup> interessant.</p>
<div class="footnotes">
<hr />
<ol>
<li id="fn:fnfeed_1">
<p>Artikel zu Miniflux: <a href="Link"><a href="https://www.robbinaer.info/index.php?post/78">https://www.robbinaer.info/index.php?post/78</a></a>&#160;<a href="#fnref1:fnfeed_1" rev="footnote" class="footnote-backref">&#8617;</a></p>
</li>
<li id="fn:fnfeed_2">
<p>OSBN: <a href="Link"><a href="https://osbn.de">https://osbn.de</a></a>&#160;<a href="#fnref1:fnfeed_2" rev="footnote" class="footnote-backref">&#8617;</a></p>
</li>
<li id="fn:fnfeed_3">
<p>Miniflux: <a href="Link"><a href="https://miniflux.net">https://miniflux.net</a></a>&#160;<a href="#fnref1:fnfeed_3" rev="footnote" class="footnote-backref">&#8617;</a></p>
</li>
<li id="fn:fnfeed_4">
<p>Fever: <a href="Link"><a href="https://feedafever.com">https://feedafever.com</a></a>&#160;<a href="#fnref1:fnfeed_4" rev="footnote" class="footnote-backref">&#8617;</a></p>
</li>
<li id="fn:fnfeed_5">
<p>wallabag: <a href="Link"><a href="https://wallabag.org/de">https://wallabag.org/de</a></a>&#160;<a href="#fnref1:fnfeed_5" rev="footnote" class="footnote-backref">&#8617;</a></p>
</li>
<li id="fn:fnfeed_6">
<p>Devonthink: <a href="Link"><a href="http://www.devontechnologies.com/de/produkte/devonthink/ueberblick.html">http://www.devontechnologies.com/de/produkte/devonthink/ueberblick.html</a></a>&#160;<a href="#fnref1:fnfeed_6" rev="footnote" class="footnote-backref">&#8617;</a></p>
</li>
<li id="fn:fnfeed_7">
<p>Podcast MacMittwoch: <a href="Link"><a href="https://itunes.apple.com/de/podcast/macmittwoch/id1212750085?mt=2ueberblick.html">https://itunes.apple.com/de/podcast/macmittwoch/id1212750085?mt=2ueberblick.html</a></a>&#160;<a href="#fnref1:fnfeed_7" rev="footnote" class="footnote-backref">&#8617;</a></p>
</li>
</ol>
</div></div>
</article>
+67
View File
@@ -0,0 +1,67 @@
<HTML>
<HEAD>
<TITLE>Alice's Adventures in Wonderland (Project Gutenberg)</TITLE>
</HEAD>
<frameset Rows="50, *">
<frame src="alice-finfo.html">
<frame src="alice-ftitle.html" name="alice-main">
</frameset>
<noframes>
<BODY>
<H1>Alice's Adventures in Wonderland</H1>
<H1>Lewis Carroll</H1>
<H1>The Millennium Fulcrum Edition 3.0</H1>
NOTE: This is a hypertext formatted version of the Project Gutenberg edition.
For more information, check the
<A HREF="alice-small.txt">small print</A>
or check out the
<A HREF="ftp://uiarchive.cso.uiuc.edu/pub/etext/gutenberg/etext91/alice30.txt">
full ascii text</A>. The original Tenniel illustrations are also available
due to the efforts of Project Gutenberg. You can if you like, grab them as a
<A HREF="ftp://uiarchive.cso.uiuc.edu/pub/etext/gutenberg/etext94/algif10.zip">
"zip file"</A> or read the <A HREF="algif-small.txt">small print</A>
that comes with them.
This document is part of a small, but growing collection of html formatted
etexts. (Others may be found in either my <A
HREF="http://www.cs.cmu.edu/Web/People/rgs/rgs-home.html">home page</A> or
John Ockerbloom's indexes by <A
HREF="http://www.cs.cmu.edu/Web/bookauthors.html">author</A> and <A
HREF="http://www.cs.cmu.edu/Web/booktitles.html">title</A>.)
I am still trying to figure out whether anyone else is interested in these
on-line readable documents. If you appreciate this document or would like to
see more such, send me mail at "rgs@cs.cmu.edu".
<P>
<A HREF="alice01a.gif"><IMG SRC="alice01th.gif"></A>
<P>
<H2>CONTENTS</H2>
<PRE>
CHAPTER I: <A HREF="alice-I.html">Down the Rabbit-Hole</A>
CHAPTER II: <A HREF="alice-II.html">The Pool of Tears</A>
CHAPTER III: <A HREF="alice-III.html">A Caucus-Race and a Long Tale</A>
CHAPTER IV: <A HREF="alice-IV.html">The Rabbit Sends in a Little Bill</A>
CHAPTER V: <A HREF="alice-V.html">Advice from a Caterpillar</A>
CHAPTER VI: <A HREF="alice-VI.html">Pig and Pepper</A>
CHAPTER VII: <A HREF="alice-VII.html">A Mad Tea-Party</A>
CHAPTER VIII: <A HREF="alice-VIII.html">The Queen's Croquet-Ground</A>
CHAPTER IX: <A HREF="alice-IX.html">The Mock Turtle's Story</A>
CHAPTER X: <A HREF="alice-X.html">The Lobster Quadrille</A>
CHAPTER XI: <A HREF="alice-XI.html">Who Stole the Tarts?</A>
CHAPTER XII: <A HREF="alice-XII.html">Alice's Evidence</A>
</PRE>
<ADDRESS><A HREF="mailto:rgs@cs.cmu.edu">Robert Stockton</A></ADDRESS>
<P>
<!- Access counter added 5/25 1:49am ->
<A href="http://www.dbasics.com/cgi-bin/pages.cgi?143205747"><IMG SRC="http://www.dbasics.com/cgi-bin/counter.cgi?143205747.2&(none)"></A> Access statistics from htmlZine
<!- This page has been visited
A HREF="http://counter.digits.com/wc?--info=yes&--name=rgsalice"
IMG SRC="http://counter.digits.com/wc/-d/4/-r/-z/rgsalice"
ALIGN=absmiddle WIDTH=60 HEIGHT=20 BORDER=0 HSPACE=4 ALT="????"/A
times since March 2, 1996. ->
</BODY>
</noframes>
</HTML>