Compare commits

...
29 Commits
Author SHA1 Message Date
Jérémy BenoistandGitHub 7cd8476d38 Merge pull request #79 from j0k3r/fix/psr-log-2-3
Allow `psr/log` 2.0 & 3.0
2022-10-17 22:44:36 +02:00
Jeremy Benoist 82083c872b Allow psr/log 2.0 & 3.0 2022-10-17 22:42:47 +02:00
Kevin DecherfandJeremy Benoist 6689f19956 Strip script and style tags through ::clean() method instead of preg_replace
Huge tags can lead to a failure of preg_replace, thus erasing the whole
fetched content.

Fixes https://github.com/wallabag/wallabag/issues/5847

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-06-13 09:13:23 +02:00
Jérémy BenoistandGitHub 0c0653dad6 Merge pull request #73 from Kdecherf/fix/impr
Fix `isPhrasingContent` conditions, text node replacement
2022-02-16 00:03:37 +01:00
Kevin Decherf 2ab87d7445 Fix isPhrasingContent conditions, text node replacement
It also disables reverting forced paragraph elements as it can break
layouts or corrupt content.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-02-15 20:53:28 +01:00
Jérémy BenoistandGitHub 8af69ad68c Merge pull request #71 from j0k3r/feature/enable-rector
Add Rector
2022-02-04 12:15:57 +01:00
Jeremy Benoist c2a1639b34 Add Rector 2022-02-04 12:13:37 +01:00
Jérémy BenoistandGitHub ccf1b336c5 Merge pull request #64 from Kdecherf/improvements 2022-02-04 05:27:57 +01:00
Kevin Decherf a44c4e5482 Add routine to remove invisible nodes
Readability was previously removing (was trying to actually, see next
section) invisible nodes using a pattern from `unlikelyCandidates`. This
was quite hacky and was removed during a backport of logics from
mozilla/readability. There is still a need to remove them so here we
are. We still use a pattern but specifically against the style
attribute. We also remove nodes with the attribute `hidden`.

The clean feature of tidy actually replaces inline style attributes
with css classes thus preventing readability to detect invisible nodes,
see https://github.com/htacg/tidy-html5/blob/5.6.0/src/clean.c#L1488
We therefore set clean configuration to false.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-02-04 01:23:11 +01:00
Kevin Decherf b580cf216d Backport some logics from mozilla/readability
This change backports several things from mozilla/readability:

- Add child score to all ancestors instead of the first parent only
- Check 5 top candidates and try to find alternative candidates within
  ancestors, this can help to find a better parent and grab more content
- Reduce patterns from `unlikelyCandidates` to the one used by Mozilla
  as ours tend to remove useful nodes
- Score headers (h2 to h6) by default in addition to div, p, td and
  section

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2022-02-04 01:05:13 +01:00
Jérémy BenoistandGitHub 2e9349f076 Merge pull request #69 from j0k3r/feature/php-7.2
Require PHP >= 7.2
2022-02-02 12:53:28 +01:00
Jeremy Benoist c4bba53dbe Remove Scrutinizer 2022-02-02 12:52:12 +01:00
Jeremy Benoist 66215a6c80 Require PHP >= 7.2
- remove test on Composer v1
- remove deprecated function
- move `loadHtml()` into `init()` instead of `__construct`

Kinda prepare 2.0 version :)
2022-02-02 12:44:24 +01:00
Jérémy BenoistandGitHub b1a20a9575 Merge pull request #68 from open-source-contributions/master
Using assertSame to make assertion equal strict
2021-12-17 12:17:44 +01:00
peter279k 97c02e8ad4 Using assertSame to make assertion equal strict 2021-12-17 19:09:36 +08:00
Jérémy BenoistandGitHub c506b7ebd7 Merge pull request #67 from j0k3r/fix/psr-log-void
Fix deprecated message
2021-11-29 21:06:47 +01:00
Jeremy Benoist d0af21814a Ditch assertContains & assertNotContains 2021-11-29 21:04:56 +01:00
Jeremy Benoist 5b1eba79bd Test on PHP 8 & drop Travis 2021-11-29 21:00:36 +01:00
Jérémy BenoistandGitHub fabf096ce6 Fix deprecated message
> Method "Psr\Log\LoggerAwareInterface::setLogger()" might add "void" as a native return type declaration in the future. Do the same in implementation "Readability\Readability" now to avoid errors or add an explicit @return annotation to suppress this message.
2021-11-29 20:50:10 +01:00
Jérémy BenoistandGitHub 8ce1663238 Merge pull request #66 from Kdecherf/fix/figure 2021-10-29 16:24:41 +02:00
Kevin Decherf eb72a315c4 Clean empty figure tags without ending
See 'Tag omission' https://developer.mozilla.org/en-US/docs/Web/HTML/Element/figure

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2021-10-29 15:36:19 +02:00
Jérémy BenoistandGitHub d5330a9c28 Merge pull request #65 from j0k3r/fix/php-cs
Fix PHPCS config file
2021-10-04 11:42:34 +02:00
Jeremy Benoist 635f8963dc Fix Travis & PHPStan 2021-10-04 11:37:33 +02:00
Jeremy Benoist 19b2a25d96 Fix PHPCS config file 2021-10-04 11:32:34 +02:00
Jérémy BenoistandGitHub 9a490fac07 Merge pull request #52 from nicofrand/master
Skip empty (empty innerHTML) nodes when grabbing article
2021-03-09 11:14:29 +01:00
Jérémy BenoistandGitHub 6f6b1f9e2b Merge pull request #62 from j0k3r/fix/avoid-wiped-body
Body can be wiped without tidy
2021-03-09 10:49:54 +01:00
Jeremy Benoist ea1368fac0 Body can be wiped without tidy
Re-create it in that case.

Also run CS-Fixer.
2021-03-08 11:59:24 +01:00
Jérémy BenoistandGitHub be81eb2a4f Create FUNDING.yml 2020-12-08 09:47:38 +01:00
nicofrand ff78c63e6d Skip empty (empty innerHTML) nodes when grabbing article 2019-05-25 16:12:52 +02:00
17 changed files with 777 additions and 650 deletions
+1
View File
@@ -6,5 +6,6 @@
/.php_cs export-ignore /.php_cs export-ignore
/phpunit.xml.dist export-ignore /phpunit.xml.dist export-ignore
/phpstan.neon export-ignore /phpstan.neon export-ignore
/rector.php export-ignore
/.github export-ignore /.github export-ignore
/tests export-ignore /tests export-ignore
+1
View File
@@ -0,0 +1 @@
github: j0k3r
+5 -11
View File
@@ -8,38 +8,32 @@ on:
branches: branches:
- master - master
env:
SYMFONY_PHPUNIT_VERSION: 7.5
jobs: jobs:
coding-standards: coding-standards:
name: "CS Fixer & PHPStan" name: "CS Fixer & PHPStan"
runs-on: "ubuntu-18.04" runs-on: "ubuntu-20.04"
strategy: strategy:
matrix: matrix:
php: php:
- "7.3" - "7.2"
steps: steps:
- name: "Checkout" - name: "Checkout"
uses: "actions/checkout@v2" uses: "actions/checkout@v3"
- name: "Install PHP" - name: "Install PHP"
uses: "shivammathur/setup-php@v2" uses: "shivammathur/setup-php@v2"
with: with:
coverage: "none" coverage: "none"
php-version: "${{ matrix.php }}" php-version: "${{ matrix.php }}"
tools: cs2pr, composer:v1 tools: cs2pr, composer:v2
ini-values: "date.timezone=Europe/Paris" ini-values: "date.timezone=Europe/Paris"
env: env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }} COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Add PHPStan"
run: "composer require phpstan/phpstan phpstan/phpstan-phpunit --dev --no-progress --no-suggest"
- name: "Install dependencies with Composer" - name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1" uses: "ramsey/composer-install@v2"
with: with:
composer-options: "--optimize-autoloader --prefer-dist" composer-options: "--optimize-autoloader --prefer-dist"
+96 -149
View File
@@ -14,163 +14,20 @@ env:
jobs: jobs:
phpunit: phpunit:
name: "PHPUnit (PHP ${{ matrix.php }})" name: "PHPUnit (PHP ${{ matrix.php }})"
runs-on: "ubuntu-18.04" runs-on: "ubuntu-20.04"
strategy: strategy:
matrix: matrix:
php: php:
- "5.6"
- "7.0"
- "7.1"
- "7.2" - "7.2"
- "7.3" - "7.3"
- "7.4" - "7.4"
- "8.0"
- "8.1"
steps: steps:
- name: "Checkout" - name: "Checkout"
uses: "actions/checkout@v2" uses: "actions/checkout@v3"
with:
fetch-depth: 2
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "none"
tools: composer:v1
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Force PHPUnit version"
if: matrix.php >= '7.2'
run: "echo $SYMFONY_PHPUNIT_VERSION"
env:
SYMFONY_PHPUNIT_VERSION: 7.5
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1"
with:
composer-options: "--optimize-autoloader --prefer-dist"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"
phpunit-coverage:
name: "PHPUnit coverage (PHP ${{ matrix.php }})"
runs-on: "ubuntu-18.04"
strategy:
matrix:
php:
- "7.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v2"
with:
fetch-depth: 2
- name: "Install PHP with Xdebug"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "xdebug"
tools: composer:v1
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1"
with:
composer-options: "--optimize-autoloader --prefer-dist"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit (with coverage)"
run: "php vendor/bin/simple-phpunit -v --coverage-clover build/logs/clover.xml"
env:
SYMFONY_PHPUNIT_VERSION: 7.5
- name: "Retrieve Coveralls phar"
run: "wget https://github.com/php-coveralls/php-coveralls/releases/download/v2.4.2/php-coveralls.phar"
- name: "Enable Coveralls phar"
run: "chmod +x php-coveralls.phar"
- name: "Upload to Coveralls"
run: "php php-coveralls.phar -v -x build/logs/clover.xml"
env:
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
phpunit-lowest:
name: "PHPUnit lowest deps (PHP ${{ matrix.php }})"
runs-on: "ubuntu-18.04"
strategy:
matrix:
php:
- "7.2"
steps:
- name: "Checkout"
uses: "actions/checkout@v2"
with:
fetch-depth: 2
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "none"
tools: composer:v1
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1"
with:
composer-options: "--optimize-autoloader --prefer-dist"
dependency-versions: "lowest"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"
env:
SYMFONY_PHPUNIT_VERSION: 7.5
phpunit-composerv2:
name: "PHPUnit with Composer v2 (PHP ${{ matrix.php }})"
runs-on: "ubuntu-18.04"
strategy:
matrix:
php:
- "7.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v2"
with: with:
fetch-depth: 2 fetch-depth: 2
@@ -189,7 +46,7 @@ jobs:
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update" run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer" - name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v1" uses: "ramsey/composer-install@v2"
with: with:
composer-options: "--optimize-autoloader --prefer-dist" composer-options: "--optimize-autoloader --prefer-dist"
@@ -198,5 +55,95 @@ jobs:
- name: "Run PHPUnit" - name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v" run: "php vendor/bin/simple-phpunit -v"
phpunit-coverage:
name: "PHPUnit coverage (PHP ${{ matrix.php }})"
runs-on: "ubuntu-20.04"
strategy:
matrix:
php:
- "7.4"
steps:
- name: "Checkout"
uses: "actions/checkout@v3"
with:
fetch-depth: 2
- name: "Install PHP with Xdebug"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "xdebug"
tools: composer:v2
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env: env:
SYMFONY_PHPUNIT_VERSION: 7.5 COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v2"
with:
composer-options: "--optimize-autoloader --prefer-dist"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit (with coverage)"
run: "php vendor/bin/simple-phpunit -v --coverage-clover build/logs/clover.xml"
- name: "Retrieve Coveralls phar"
run: "wget https://github.com/php-coveralls/php-coveralls/releases/download/v2.4.2/php-coveralls.phar"
- name: "Enable Coveralls phar"
run: "chmod +x php-coveralls.phar"
- name: "Upload to Coveralls"
run: "php php-coveralls.phar -v -x build/logs/clover.xml"
env:
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
phpunit-lowest:
name: "PHPUnit lowest deps (PHP ${{ matrix.php }})"
runs-on: "ubuntu-20.04"
strategy:
matrix:
php:
- "7.2"
steps:
- name: "Checkout"
uses: "actions/checkout@v3"
with:
fetch-depth: 2
- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "none"
tools: composer:v2
extensions: tidy
ini-values: "date.timezone=Europe/Paris"
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: "Remove useless deps"
run: "composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update"
- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v2"
with:
composer-options: "--optimize-autoloader --prefer-dist"
dependency-versions: "lowest"
- name: "Setup logs"
run: "mkdir -p build/logs"
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"
+1
View File
@@ -2,4 +2,5 @@ vendor/
coverage/ coverage/
composer.lock composer.lock
.php_cs.cache .php_cs.cache
.php-cs-fixer.cache
.phpunit.result.cache .phpunit.result.cache
+8 -7
View File
@@ -1,6 +1,11 @@
<?php <?php
return PhpCsFixer\Config::create() $finder = (new PhpCsFixer\Finder())
->in(__DIR__)
->exclude(['vendor', 'var', 'web'])
;
return (new PhpCsFixer\Config())
->setUsingCache(true) ->setUsingCache(true)
->setRiskyAllowed(true) ->setRiskyAllowed(true)
->setRules([ ->setRules([
@@ -9,7 +14,7 @@ return PhpCsFixer\Config::create()
'array_syntax' => ['syntax' => 'short'], 'array_syntax' => ['syntax' => 'short'],
'combine_consecutive_unsets' => true, 'combine_consecutive_unsets' => true,
'heredoc_to_nowdoc' => true, 'heredoc_to_nowdoc' => true,
'no_extra_consecutive_blank_lines' => ['break', 'continue', 'extra', 'return', 'throw', 'use', 'parenthesis_brace_block', 'square_brace_block', 'curly_brace_block'], 'no_extra_blank_lines' => ['tokens' => ['break', 'continue', 'extra', 'return', 'throw', 'use', 'parenthesis_brace_block', 'square_brace_block', 'curly_brace_block']],
'no_unreachable_default_argument_value' => true, 'no_unreachable_default_argument_value' => true,
'no_useless_else' => true, 'no_useless_else' => true,
'no_useless_return' => true, 'no_useless_return' => true,
@@ -22,9 +27,5 @@ return PhpCsFixer\Config::create()
'strict_param' => true, 'strict_param' => true,
'concat_space' => ['spacing' => 'one'], 'concat_space' => ['spacing' => 'one'],
]) ])
->setFinder( ->setFinder($finder)
PhpCsFixer\Finder::create()
->exclude(['vendor'])
->in(__DIR__)
)
; ;
-12
View File
@@ -1,12 +0,0 @@
tools:
external_code_coverage: false
build:
nodes:
analysis:
tests:
override:
- php-scrutinizer-run
environment:
php:
version: 7.2
-34
View File
@@ -1,34 +0,0 @@
dist: xenial
os: linux
language: php
# avoid double Travis build when the PR is created on upstream
if: |
type = pull_request OR \
branch = master
jobs:
include:
- php: 8.0
env: SYMFONY_PHPUNIT_VERSION=7.5
fast_finish: true
allow_failures:
- php: 8.0
cache:
directories:
- $HOME/.composer/cache/files
before_install:
- if [ -n "$GH_TOKEN" ]; then composer config github-oauth.github.com ${GH_TOKEN}; fi;
install:
- if [ "$CS_FIXER" != "run" ]; then composer remove friendsofphp/php-cs-fixer --dev --no-progress --no-update ; fi;
- composer update --prefer-dist --no-progress --no-suggest -o $
- php vendor/bin/simple-phpunit install
before_script:
- mkdir -p build/logs
script:
- php vendor/bin/simple-phpunit -v
+1 -2
View File
@@ -1,12 +1,11 @@
# Readability # Readability
![CI](https://github.com/j0k3r/php-readability/workflows/CI/badge.svg) ![CI](https://github.com/j0k3r/php-readability/workflows/CI/badge.svg)
[![Build Status](https://travis-ci.org/j0k3r/php-readability.svg?branch=master)](https://travis-ci.org/j0k3r/php-readability)
[![Coverage Status](https://coveralls.io/repos/j0k3r/php-readability/badge.svg?branch=master&service=github)](https://coveralls.io/github/j0k3r/php-readability/?branch=master) [![Coverage Status](https://coveralls.io/repos/j0k3r/php-readability/badge.svg?branch=master&service=github)](https://coveralls.io/github/j0k3r/php-readability/?branch=master)
[![Total Downloads](https://poser.pugx.org/j0k3r/php-readability/downloads)](https://packagist.org/packages/j0k3r/php-readability) [![Total Downloads](https://poser.pugx.org/j0k3r/php-readability/downloads)](https://packagist.org/packages/j0k3r/php-readability)
[![License](https://poser.pugx.org/j0k3r/php-readability/license)](https://packagist.org/packages/j0k3r/php-readability) [![License](https://poser.pugx.org/j0k3r/php-readability/license)](https://packagist.org/packages/j0k3r/php-readability)
This is an extract of the Readability class from this [full-text-rss](https://github.com/Dither/full-text-rss) fork. It can be defined as a better version of the original [php-readability](https://bitbucket.org/fivefilters/php-readability/overview). This is an extract of the Readability class from this [full-text-rss](https://github.com/Dither/full-text-rss) fork. It can be defined as a better version of the original [php-readability](https://bitbucket.org/fivefilters/php-readability).
## Differences ## Differences
+7 -4
View File
@@ -24,15 +24,18 @@
"role": "Developer (original JS version)" "role": "Developer (original JS version)"
}], }],
"require": { "require": {
"php": ">=5.6.0", "php": ">=7.2.0",
"ext-mbstring": "*", "ext-mbstring": "*",
"psr/log": "^1.0", "psr/log": "^1.0.1 || ^2.0 || ^3.0",
"masterminds/html5": "^2.7" "masterminds/html5": "^2.7"
}, },
"require-dev": { "require-dev": {
"friendsofphp/php-cs-fixer": "^2.14", "friendsofphp/php-cs-fixer": "^3.0",
"monolog/monolog": "^1.24|^2.1", "monolog/monolog": "^1.24|^2.1",
"symfony/phpunit-bridge": "^4.2.3" "symfony/phpunit-bridge": "^4.4|^5.3|^6.0",
"phpstan/phpstan": "^1.3",
"phpstan/phpstan-phpunit": "^1.0",
"rector/rector": "^0.12.15"
}, },
"suggest": { "suggest": {
"ext-tidy": "Used to clean up given HTML and to avoid problems with bad HTML structure." "ext-tidy": "Used to clean up given HTML and to avoid problems with bad HTML structure."
+3 -1
View File
@@ -6,7 +6,9 @@ parameters:
# https://github.com/phpstan/phpstan/issues/694#issuecomment-350724288 # https://github.com/phpstan/phpstan/issues/694#issuecomment-350724288
bootstrapFiles: bootstrapFiles:
- vendor/bin/.phpunit/phpunit-7.5-0/vendor/autoload.php - vendor/bin/.phpunit/phpunit-8.5-0/vendor/autoload.php
checkMissingIterableValueType: false
includes: includes:
- vendor/phpstan/phpstan-phpunit/extension.neon - vendor/phpstan/phpstan-phpunit/extension.neon
+1 -1
View File
@@ -11,7 +11,7 @@
> >
<testsuites> <testsuites>
<testsuite name="Readability Test Suite"> <testsuite name="Readability">
<directory>./tests/</directory> <directory>./tests/</directory>
</testsuite> </testsuite>
</testsuites> </testsuites>
+32
View File
@@ -0,0 +1,32 @@
<?php
declare(strict_types=1);
use Rector\Core\Configuration\Option;
use Rector\Core\ValueObject\PhpVersion;
use Rector\Set\ValueObject\LevelSetList;
use Symfony\Component\DependencyInjection\Loader\Configurator\ContainerConfigurator;
return static function (ContainerConfigurator $containerConfigurator): void {
$parameters = $containerConfigurator->parameters();
// paths to refactor; solid alternative to CLI arguments
$parameters->set(Option::PATHS, [
__DIR__ . '/src',
__DIR__ . '/tests',
]);
// Path to phpstan with extensions, that PHPSTan in Rector uses to determine types
$parameters->set(Option::PHPSTAN_FOR_RECTOR_PATH, __DIR__ . '/phpstan.neon');
$parameters->set(Option::BOOTSTRAP_FILES, [
__DIR__ . '/vendor/bin/.phpunit/phpunit-8.5-0/vendor/autoload.php',
__DIR__ . '/vendor/autoload.php',
]);
// Define what rule sets will be applied
$containerConfigurator->import(LevelSetList::UP_TO_PHP_72);
// is your PHP version different from the one your refactor to?
$parameters->set(Option::PHP_VERSION_FEATURES, PhpVersion::PHP_72);
};
+6 -2
View File
@@ -47,15 +47,17 @@ class JSLikeHTMLElement extends \DOMElement
{ {
if ('innerHTML' !== $name) { if ('innerHTML' !== $name) {
$trace = debug_backtrace(); $trace = debug_backtrace();
trigger_error('Undefined property via __set(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], E_USER_NOTICE); trigger_error('Undefined property via __set(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], \E_USER_NOTICE);
return; return;
} }
// first, empty the element // first, empty the element
if (isset($this->childNodes)) {
for ($x = $this->childNodes->length - 1; $x >= 0; --$x) { for ($x = $this->childNodes->length - 1; $x >= 0; --$x) {
$this->removeChild($this->childNodes->item($x)); $this->removeChild($this->childNodes->item($x));
} }
}
// $value holds our new inner HTML // $value holds our new inner HTML
$value = trim($value); $value = trim($value);
@@ -112,15 +114,17 @@ class JSLikeHTMLElement extends \DOMElement
if ('innerHTML' === $name) { if ('innerHTML' === $name) {
$inner = ''; $inner = '';
if (isset($this->childNodes)) {
foreach ($this->childNodes as $child) { foreach ($this->childNodes as $child) {
$inner .= $this->ownerDocument->saveXML($child); $inner .= $this->ownerDocument->saveXML($child);
} }
}
return $inner; return $inner;
} }
$trace = debug_backtrace(); $trace = debug_backtrace();
trigger_error('Undefined property via __get(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], E_USER_NOTICE); trigger_error('Undefined property via __get(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], \E_USER_NOTICE);
} }
public function __toString() public function __toString()
+324 -262
View File
File diff suppressed because it is too large Load Diff
+202 -143
View File
@@ -4,120 +4,129 @@ namespace Tests\Readability;
use Monolog\Handler\TestHandler; use Monolog\Handler\TestHandler;
use Monolog\Logger; use Monolog\Logger;
use Psr\Log\LoggerInterface;
use Readability\JSLikeHTMLElement;
use Readability\Readability; use Readability\Readability;
class ReadabilityTest extends \PHPUnit\Framework\TestCase class ReadabilityTest extends \PHPUnit\Framework\TestCase
{ {
/** @var TestHandler */
public $logHandler; public $logHandler;
/** @var LoggerInterface */
public $logger; public $logger;
/** /**
* @requires extension tidy * @requires extension tidy
*/ */
public function testConstructDefault() public function testConstructDefault(): void
{ {
$readability = $this->getReadability(''); $readability = $this->getReadability('');
$readability->init();
$this->assertNull($readability->url); $this->assertNull($readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom); $this->assertInstanceOf('DomDocument', $readability->dom);
} }
public function testConstructHtml5Parser() public function testConstructHtml5Parser(): void
{ {
$readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'html5lib'); $readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'html5lib');
$readability->init();
$this->assertEquals('http://0.0.0.0', $readability->url); $this->assertSame('http://0.0.0.0', $readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom); $this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertEquals('<html/>', $readability->original_html); $this->assertSame('<html/>', $readability->original_html);
} }
/** /**
* @requires extension tidy * @requires extension tidy
*/ */
public function testConstructSimple() public function testConstructSimple(): void
{ {
$readability = $this->getReadability('<html/>', 'http://0.0.0.0'); $readability = $this->getReadability('<html/>', 'http://0.0.0.0');
$readability->init();
$this->assertEquals('http://0.0.0.0', $readability->url); $this->assertSame('http://0.0.0.0', $readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom); $this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertEquals('<html/>', $readability->original_html); $this->assertSame('<html/>', $readability->original_html);
$this->assertTrue($readability->tidied); $this->assertTrue($readability->tidied);
} }
public function testConstructDefaultWithoutTidy() public function testConstructDefaultWithoutTidy(): void
{ {
$readability = $this->getReadability('', null, 'libxml', false); $readability = $this->getReadability('', null, 'libxml', false);
$readability->init();
$this->assertNull($readability->url); $this->assertNull($readability->url);
$this->assertEquals('', $readability->original_html); $this->assertSame('', $readability->original_html);
$this->assertFalse($readability->tidied); $this->assertFalse($readability->tidied);
$this->assertInstanceOf('DomDocument', $readability->dom); $this->assertInstanceOf('DomDocument', $readability->dom);
} }
public function testConstructSimpleWithoutTidy() public function testConstructSimpleWithoutTidy(): void
{ {
$readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'libxml', false); $readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'libxml', false);
$readability->init();
$this->assertEquals('http://0.0.0.0', $readability->url); $this->assertSame('http://0.0.0.0', $readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom); $this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertEquals('<html/>', $readability->original_html); $this->assertSame('<html/>', $readability->original_html);
$this->assertFalse($readability->tidied); $this->assertFalse($readability->tidied);
} }
public function testInitNoContent() public function testInitNoContent(): void
{ {
$readability = $this->getReadability('<html/>', 'http://0.0.0.0'); $readability = $this->getReadability('<html/>', 'http://0.0.0.0');
$res = $readability->init(); $res = $readability->init();
$this->assertFalse($res); $this->assertFalse($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('Sorry, Readability was unable to parse this page for content.', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('Sorry, Readability was unable to parse this page for content.', $readability->getContent()->getInnerHtml());
} }
public function testInitP() public function testInitP(): void
{ {
$readability = $this->getReadability(str_repeat('<p>This is the awesome content :)</p>', 7), 'http://0.0.0.0'); $readability = $this->getReadability(str_repeat('<p>This is the awesome content :)</p>', 7), 'http://0.0.0.0');
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is the awesome content :)', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is the awesome content :)', $readability->getContent()->getInnerHtml());
} }
public function testInitDivP() public function testInitDivP(): void
{ {
$readability = $this->getReadability('<div>' . str_repeat('<p>This is the awesome content :)</p>', 7) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('<p>This is the awesome content :)</p>', 7) . '</div>', 'http://0.0.0.0');
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is the awesome content :)', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is the awesome content :)', $readability->getContent()->getInnerHtml());
} }
public function testInitDiv() public function testInitDiv(): void
{ {
$readability = $this->getReadability('<div>' . str_repeat('This is the awesome content :)', 7) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('This is the awesome content :)', 7) . '</div>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is the awesome content :)', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is the awesome content :)', $readability->getContent()->getInnerHtml());
} }
public function testWithFootnotes() public function testWithFootnotes(): void
{ {
$readability = $this->getReadability('<div>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '</div>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
@@ -125,16 +134,16 @@ class ReadabilityTest extends \PHPUnit\Framework\TestCase
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('readabilityFootnoteLink', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('readabilityFootnoteLink', $readability->getContent()->getInnerHtml());
$this->assertContains('readabilityLink-3', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('readabilityLink-3', $readability->getContent()->getInnerHtml());
} }
public function testStandardClean() public function testStandardClean(): void
{ {
$readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<a href="#nofollow" rel="nofollow">will NOT be removed</a></div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<a href="#nofollow" rel="nofollow">will NOT be removed</a></div>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
@@ -142,76 +151,76 @@ class ReadabilityTest extends \PHPUnit\Framework\TestCase
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('will NOT be removed', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('will NOT be removed', $readability->getContent()->getInnerHtml());
$this->assertNotContains('<h2>', $readability->getContent()->getInnerHtml()); $this->assertStringNotContainsString('<h2>', $readability->getContent()->getInnerHtml());
} }
public function testWithIframe() public function testWithIframe(): void
{ {
$readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe><iframe>http://soundcloud.com/test</iframe></p></div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe><iframe>http://soundcloud.com/test</iframe></p></div>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<div readability=', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('nofollow', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('nofollow', $readability->getContent()->getInnerHtml());
} }
public function testWithArticle() public function testWithArticle(): void
{ {
$readability = $this->getReadability('<article><p>' . str_repeat('This is an awesome text with some links, here there are: the awesome', 20) . '</p><p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article><p>' . str_repeat('This is an awesome text with some links, here there are: the awesome', 20) . '</p><p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe></p></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('nofollow', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('nofollow', $readability->getContent()->getInnerHtml());
} }
public function testWithAside() public function testWithAside(): void
{ {
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<footer><aside>' . str_repeat('<p>This is an awesome text with some links, here there are</p>', 8) . '</aside></footer></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<footer><aside>' . str_repeat('<p>This is an awesome text with some links, here there are</p>', 8) . '</aside></footer></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('<aside>', $readability->getContent()->getInnerHtml()); $this->assertStringNotContainsString('<aside>', $readability->getContent()->getInnerHtml());
$this->assertContains('<footer readability="5"/>', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<footer readability="9"/>', $readability->getContent()->getInnerHtml());
} }
public function testWithClasses() public function testWithClasses(): void
{ {
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text should be removed', $readability->getContent()->getInnerHtml()); $this->assertStringNotContainsString('This text should be removed', $readability->getContent()->getInnerHtml());
} }
public function testWithClassesWithoutLightClean() public function testWithClassesWithoutLightClean(): void
{ {
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
@@ -219,122 +228,119 @@ class ReadabilityTest extends \PHPUnit\Framework\TestCase
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('alt="article"', $readability->getContent()->getInnerHtml());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text should be removed', $readability->getContent()->getInnerHtml()); $this->assertStringNotContainsString('This text should be removed', $readability->getContent()->getInnerHtml());
} }
public function testWithTd() public function testWithTd(): void
{ {
$readability = $this->getReadability('<table><tr>' . str_repeat('<td><p>This is an awesome text with some links, here there are the awesome</td>', 7) . '</tr></table>', 'http://0.0.0.0'); $readability = $this->getReadability('<table><tr>' . str_repeat('<td><p>This is an awesome text with some links, here there are the awesome</td>', 7) . '</tr></table>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
} }
public function testWithSameClasses() public function testWithSameClasses(): void
{ {
$readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<div class="awesomecontent">This text is also an awesome text and you should know that !</div></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<div class="awesomecontent">This text is also an awesome text and you should know that !</div></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testWithScript() public function testWithScript(): void
{ {
$readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p><script>This text is also an awesome text and you should know that !</script></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p><script>This text is also an awesome text and you should know that !</script></p></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEmpty($readability->getTitle()->getInnerHtml()); $this->assertEmpty($readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml()); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testTitle() public function testTitle(): void
{ {
$readability = $this->getReadability('<title>this is my title</title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<title>this is my title</title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEquals('this is my title', $readability->getTitle()->getInnerHtml()); $this->assertSame('this is my title', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml()); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testTitleWithDash() public function testTitleWithDash(): void
{ {
$readability = $this->getReadability('<title> title2 - title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<title> title2 - title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEquals('title2 - title3', $readability->getTitle()->getInnerHtml()); $this->assertSame('title2 - title3', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml()); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testTitleWithDoubleDot() public function testTitleWithDoubleDot(): void
{ {
$readability = $this->getReadability('<title> title2 : title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<title> title2 : title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEquals('title2 : title3', $readability->getTitle()->getInnerHtml()); $this->assertSame('title2 : title3', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml()); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testTitleTooShortUseH1() public function testTitleTooShortUseH1(): void
{ {
$readability = $this->getReadability('<title>too short</title><h1>this is my h1 title !</h1><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0'); $readability = $this->getReadability('<title>too short</title><h1>this is my h1 title !</h1><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertEquals('this is my h1 title !', $readability->getTitle()->getInnerHtml()); $this->assertSame('this is my h1 title !', $readability->getTitle()->getInnerHtml());
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This is an awesome text with some links, here there are', $readability->getContent()->getInnerHtml());
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml()); $this->assertStringNotContainsString('This text is also an awesome text and you should know that', $readability->getContent()->getInnerHtml());
} }
public function testAutoClosingIframeNotThrowingException(): void
{
error_reporting(\E_ALL | \E_STRICT);
ini_set('display_errors', '1');
// dummy function to be used to the next test // dummy function to be used to the next test
public function error2Exception($code, $string, $file, $line, $context) set_error_handler(function (int $errno, string $errstr, string $errfile, int $errline, array $errcontext) {
{ throw new \Exception($errstr, $errno);
throw new \Exception($string, $code); }, \E_ALL | \E_STRICT);
}
public function testAutoClosingIframeNotThrowingException()
{
error_reporting(E_ALL | E_STRICT);
ini_set('display_errors', true);
set_error_handler([$this, 'error2Exception'], E_ALL | E_STRICT);
$data = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> $data = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="ru-RU" prefix="og: http://ogp.me/ns#"> <html xmlns="http://www.w3.org/1999/xhtml" lang="ru-RU" prefix="og: http://ogp.me/ns#">
@@ -367,16 +373,16 @@ class ReadabilityTest extends \PHPUnit\Framework\TestCase
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
$this->assertContains('<iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"> </iframe>', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"> </iframe>', $readability->getContent()->getInnerHtml());
$this->assertContains('3D Touch', $readability->getTitle()->getInnerHtml()); $this->assertStringContainsString('3D Touch', $readability->getTitle()->getInnerHtml());
} }
/** /**
* This should generate an Exception "DOMElement::setAttribute(): ID post-60 already defined". * This should generate an Exception "DOMElement::setAttribute(): ID post-60 already defined".
*/ */
public function testAppendIdAlreadyHere() public function testAppendIdAlreadyHere(): void
{ {
$data = '<!DOCTYPE html> $data = '<!DOCTYPE html>
<html lang="fr"> <html lang="fr">
@@ -428,11 +434,11 @@ class ReadabilityTest extends \PHPUnit\Framework\TestCase
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle()); $this->assertInstanceOf(JSLikeHTMLElement::class, $readability->getTitle());
} }
public function testPostFilters() public function testPostFilters(): void
{ {
$readability = $this->getReadability('<div>' . str_repeat('<p>This <strong>is</strong> the awesome content :)</p>', 10) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('<p>This <strong>is</strong> the awesome content :)</p>', 10) . '</div>', 'http://0.0.0.0');
$readability->addPostFilter('!<strong[^>]*>(.*?)</strong>!is', ''); $readability->addPostFilter('!<strong[^>]*>(.*?)</strong>!is', '');
@@ -440,26 +446,24 @@ class ReadabilityTest extends \PHPUnit\Framework\TestCase
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertContains('This the awesome content :)', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This the awesome content :)', $readability->getContent()->getInnerHtml());
} }
public function testPreFilters() public function testPreFilters(): void
{ {
$this->markTestSkipped('Won\'t work until loadHtml() is moved in init() instead of __construct()');
$readability = $this->getReadability('<div>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>', 'http://0.0.0.0'); $readability = $this->getReadability('<div>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>', 'http://0.0.0.0');
$readability->addPreFilter('!<b[^>]*>(.*?)</b>!is', ''); $readability->addPreFilter('!<b[^>]*>(.*?)</b>!is', '');
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertContains('This the awesome and WONDERFUL content :)', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('This the awesome and WONDERFUL content :)', $readability->getContent()->getInnerHtml());
} }
public function testChildNodeGoneNull() public function testChildNodeGoneNull(): void
{ {
// from http://www.ayyaantuu.net/ethiopia-targets-opposition-lawmakers/ // from http://www.ayyaantuu.net/ethiopia-targets-opposition-lawmakers/
$html = file_get_contents('tests/fixtures/childNodeGoesNull.html'); $html = (string) file_get_contents('tests/fixtures/childNodeGoesNull.html');
$readability = $this->getReadability($html, 'http://0.0.0.0'); $readability = $this->getReadability($html, 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
@@ -469,21 +473,76 @@ class ReadabilityTest extends \PHPUnit\Framework\TestCase
$this->assertTrue($res); $this->assertTrue($res);
} }
public function testKeepFootnotes() public function testKeepFootnotes(): void
{ {
// from https://www.schreibdichte.de/blog/feed-aggregator-und-spaeter-lesen-dienst-im-team // from https://www.schreibdichte.de/blog/feed-aggregator-und-spaeter-lesen-dienst-im-team
$html = file_get_contents('tests/fixtures/keepFootnotes.html'); $html = (string) file_get_contents('tests/fixtures/keepFootnotes.html');
$readability = $this->getReadability($html, 'http://0.0.0.0'); $readability = $this->getReadability($html, 'http://0.0.0.0');
$readability->debug = true; $readability->debug = true;
$res = $readability->init(); $res = $readability->init();
$this->assertTrue($res); $this->assertTrue($res);
$this->assertContains('<sup id="fnref1:fnfeed_2"><a href="#fn:fnfeed_2" class="footnote-ref">2</a></sup>', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<sup id="fnref1:fnfeed_2"><a href="#fn:fnfeed_2" class="footnote-ref">2</a></sup>', $readability->getContent()->getInnerHtml());
$this->assertContains('<a href="#fnref1:fnfeed_2" rev="footnote"', $readability->getContent()->getInnerHtml()); $this->assertStringContainsString('<a href="#fnref1:fnfeed_2" rev="footnote"', $readability->getContent()->getInnerHtml());
} }
private function getReadability($html, $url = null, $parser = 'libxml', $useTidy = true) public function testWithWipedBody(): void
{
// from https://www.cs.cmu.edu/~rgs/alice-table.html
$html = (string) file_get_contents('tests/fixtures/wipedBody.html');
$readability = $this->getReadability($html, 'http://0.0.0.0', 'libxml', false);
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertStringContainsString('<a href="alice-I.html">Down the Rabbit-Hole</a>', $readability->getContent()->getInnerHtml());
}
public function dataForVisibleNode(): array
{
return [
'visible node' => [
'<div>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
true,
],
'display=none' => [
'<div style="display:none;">' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
false,
],
'display=inline' => [
'<div style="display:inline;">' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
true,
],
'hidden attribute' => [
'<div hidden>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
false,
],
'missing display' => [
'<div style="color:#ccc;">' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>',
true,
],
];
}
/**
* @dataProvider dataForVisibleNode
*/
public function testVisibleNode(string $content, bool $shouldBeVisible): void
{
$readability = $this->getReadability($content, 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
if ($shouldBeVisible) {
$this->assertStringContainsString('WONDERFUL content', $readability->getContent()->getInnerHtml());
} else {
$this->assertStringNotContainsString('WONDERFUL content', $readability->getContent()->getInnerHtml());
}
}
private function getReadability(string $html, string $url = null, string $parser = 'libxml', bool $useTidy = true): Readability
{ {
$readability = new Readability($html, $url, $parser, $useTidy); $readability = new Readability($html, $url, $parser, $useTidy);
+67
View File
@@ -0,0 +1,67 @@
<HTML>
<HEAD>
<TITLE>Alice's Adventures in Wonderland (Project Gutenberg)</TITLE>
</HEAD>
<frameset Rows="50, *">
<frame src="alice-finfo.html">
<frame src="alice-ftitle.html" name="alice-main">
</frameset>
<noframes>
<BODY>
<H1>Alice's Adventures in Wonderland</H1>
<H1>Lewis Carroll</H1>
<H1>The Millennium Fulcrum Edition 3.0</H1>
NOTE: This is a hypertext formatted version of the Project Gutenberg edition.
For more information, check the
<A HREF="alice-small.txt">small print</A>
or check out the
<A HREF="ftp://uiarchive.cso.uiuc.edu/pub/etext/gutenberg/etext91/alice30.txt">
full ascii text</A>. The original Tenniel illustrations are also available
due to the efforts of Project Gutenberg. You can if you like, grab them as a
<A HREF="ftp://uiarchive.cso.uiuc.edu/pub/etext/gutenberg/etext94/algif10.zip">
"zip file"</A> or read the <A HREF="algif-small.txt">small print</A>
that comes with them.
This document is part of a small, but growing collection of html formatted
etexts. (Others may be found in either my <A
HREF="http://www.cs.cmu.edu/Web/People/rgs/rgs-home.html">home page</A> or
John Ockerbloom's indexes by <A
HREF="http://www.cs.cmu.edu/Web/bookauthors.html">author</A> and <A
HREF="http://www.cs.cmu.edu/Web/booktitles.html">title</A>.)
I am still trying to figure out whether anyone else is interested in these
on-line readable documents. If you appreciate this document or would like to
see more such, send me mail at "rgs@cs.cmu.edu".
<P>
<A HREF="alice01a.gif"><IMG SRC="alice01th.gif"></A>
<P>
<H2>CONTENTS</H2>
<PRE>
CHAPTER I: <A HREF="alice-I.html">Down the Rabbit-Hole</A>
CHAPTER II: <A HREF="alice-II.html">The Pool of Tears</A>
CHAPTER III: <A HREF="alice-III.html">A Caucus-Race and a Long Tale</A>
CHAPTER IV: <A HREF="alice-IV.html">The Rabbit Sends in a Little Bill</A>
CHAPTER V: <A HREF="alice-V.html">Advice from a Caterpillar</A>
CHAPTER VI: <A HREF="alice-VI.html">Pig and Pepper</A>
CHAPTER VII: <A HREF="alice-VII.html">A Mad Tea-Party</A>
CHAPTER VIII: <A HREF="alice-VIII.html">The Queen's Croquet-Ground</A>
CHAPTER IX: <A HREF="alice-IX.html">The Mock Turtle's Story</A>
CHAPTER X: <A HREF="alice-X.html">The Lobster Quadrille</A>
CHAPTER XI: <A HREF="alice-XI.html">Who Stole the Tarts?</A>
CHAPTER XII: <A HREF="alice-XII.html">Alice's Evidence</A>
</PRE>
<ADDRESS><A HREF="mailto:rgs@cs.cmu.edu">Robert Stockton</A></ADDRESS>
<P>
<!- Access counter added 5/25 1:49am ->
<A href="http://www.dbasics.com/cgi-bin/pages.cgi?143205747"><IMG SRC="http://www.dbasics.com/cgi-bin/counter.cgi?143205747.2&(none)"></A> Access statistics from htmlZine
<!- This page has been visited
A HREF="http://counter.digits.com/wc?--info=yes&--name=rgsalice"
IMG SRC="http://counter.digits.com/wc/-d/4/-r/-z/rgsalice"
ALIGN=absmiddle WIDTH=60 HEIGHT=20 BORDER=0 HSPACE=4 ALT="????"/A
times since March 2, 1996. ->
</BODY>
</noframes>
</HTML>