Compare commits

..
43 Commits
Author SHA1 Message Date
Jeremy BenoistandGitHub 47ce4fb7f9 Merge pull request #16 from j0k3r/info
“info” is too agressive in unlikelyCandidates
2016-10-03 09:13:35 +02:00
Jeremy Benoist 3de4e918b4 Convert header & section to p
And took `pre` element in score
2016-10-02 14:55:52 +02:00
Jeremy Benoist 5182d6cb11 “info” is too agressive in unlikelyCandidates
Some contents have a `infocontent` node (ot sth different) and they are real content.
Using only `info` as regex is too agressive and remove legitimate content.
Matching the whole word `info` (or `infos`) should be a better choice
2016-10-02 14:49:43 +02:00
Jeremy BenoistandGitHub fb9810a827 Merge pull request #15 from j0k3r/cs
Enable php-cs-fixer
2016-06-24 09:36:50 +02:00
Jeremy Benoist 22aa4c3b31 Fix composer & TLS 2016-06-23 07:44:54 +02:00
Jeremy Benoist 2ef400bf73 Enable php-cs-fixer 2016-06-23 07:28:10 +02:00
Jeremy Benoist dec4514c00 Merge pull request #14 from j0k3r/revert-bc
Revert BC changes
2016-03-01 15:14:36 +01:00
Jeremy Benoist 00f622e9b7 Revert BC changes
- avoid method signature update
- revert moving logic out of the constructor
2016-03-01 15:07:32 +01:00
Jeremy Benoist 7b47e2f1de Merge pull request #13 from j0k3r/monolog
Use Monolog instead of custom solution
2016-03-01 14:09:42 +01:00
Jeremy Benoist c756ec067e Fix tests
`getInnerText` might receive a null DOMElement if the xpath or query return no element.
2016-02-29 13:09:16 +01:00
Jeremy Benoist 8ab7d76cd5 Use Monolog instead of custom solution
Remove that ugly `openlog` & `syslog`
2016-02-29 13:01:40 +01:00
Jeremy Benoist 149a333b40 Remove addPreFilter
Pre filters are used in the __construct so adding more pre filters once the object is instantiated is useless.
2016-02-29 12:10:56 +01:00
Jeremy Benoist 209c404d7b Fix instanceof DOMElement
We previously checked `instanceof DOMElement` which was wrong since we
are in the namespace class, the class `Readability\DOMElement` does not
exists.
2016-02-29 11:53:03 +01:00
Jeremy Benoist 6e3f2e8c0b Merge pull request #12 from j0k3r/travis-coveralls
Switch to Coveralls
2016-02-29 11:18:53 +01:00
Jeremy Benoist 8cbecd7e99 Use cache for Travis
And also avoid Github api limit
2016-02-29 11:13:43 +01:00
Jeremy Benoist 438699ab15 Disable html coverage
Required only on local.
2016-02-29 11:13:15 +01:00
Jeremy Benoist ea7a62d352 Switch to Coveralls 2016-02-29 11:13:15 +01:00
Jeremy Benoist 9181fd1a2c Merge pull request #11 from j0k3r/append-id
Avoid adding id that might already exists
2016-02-29 11:12:52 +01:00
Jeremy Benoist 2951936e00 CS & PHPDoc 2016-02-29 11:00:45 +01:00
Jeremy Benoist 92965f76cf Fix composer TLS exception on PHP 5.3.3 2016-02-29 10:30:33 +01:00
Jeremy Benoist 850ade16b6 Cleanup 2016-02-29 10:22:07 +01:00
Jeremy Benoist dc590542f0 Avoid adding id that might already exists
We append a new node when it isn't a `div` or `p` (like when it's an `article`) with the same id which generate a DOM error "DOMElement::setAttribute(): ID blabla already defined".
2016-02-29 10:21:52 +01:00
Jeremy Benoist 41d7440c6e Merge pull request #10 from j0k3r/remove-tidy-constraint
Remove tidy from requirement
2015-11-10 09:55:29 +01:00
Jeremy Benoist 252bb4ef42 Add Tidy requirement in README 2015-11-09 20:25:39 +01:00
Jeremy Benoist 7c30d76b6e Ensure tests are running without Tidy 2015-11-09 19:59:18 +01:00
Jeremy Benoist 111cb08034 Improve negative element
- add unlikelyCandidates: head
- add negative: recommend
2015-11-09 19:44:11 +01:00
Jeremy Benoist 74fbf6f009 Remove tidy from requirement
Sadly we can't suggest user to install the extension
2015-11-09 11:12:07 +01:00
Jeremy Benoist 1565a6819a Add .gitattributes to slim down composer packages
When composer is used as a dependency manager for large web apps, transferring lots of extra files (such as tests and documentation) can be very time and bandwidth consuming. By using .gitattributes we can tell composer what files to ignore when distributing through composer.
2015-10-29 13:56:13 +01:00
Jeremy Benoist f71c3a4196 Do not remove html tag attributes
They might contains useful information (at least language)
2015-09-23 21:09:38 +02:00
Jeremy Benoist 255a2fc7bc Merge pull request #8 from j0k3r/nofollow
Do not remove nofollow links
2015-09-23 17:58:32 +02:00
Jeremy Benoist b77876b30a Do not remove nofollow links
Most the time, they can be usefull.
At least, it'll be a link to something unrelated. But we won't lose a link inside the content.

Also, adding some extra space.
2015-09-22 19:25:57 +02:00
Jeremy Benoist 1830dc45d4 Merge pull request #7 from j0k3r/fix-nbsp
Avoid error with  
2015-09-20 21:05:55 +02:00
Jeremy Benoist 6be1f9b984 Fix link to fivefilters fork 2015-09-18 19:19:26 +02:00
Jeremy Benoist 175196d6c2 Avoid error with  
Fix #5
2015-09-18 19:10:48 +02:00
Jeremy Benoist 2b5af601d5 Do not format output to avoid breaking apps
It'll require to jump to 2.0.0 and I think it's to soon
2015-09-15 22:25:08 +02:00
Jeremy Benoist d01eb2ac1e Use class instead of id to avoid error
It generates error like `ID XXX already defined`
2015-09-14 21:49:40 +02:00
Jeremy Benoist c5a4a490e1 CS 2015-08-24 11:10:54 +02:00
Jeremy Benoist 908a49824f Add test on title 2015-08-24 11:09:47 +02:00
Jeremy Benoist c67189248e Backport changes from wallabag
https://github.com/wallabag/php-readability/commit/e9e4ff87f8fc56d406ccdd5a9a7f1d3d6af07e79
2015-08-24 11:09:38 +02:00
Jeremy Benoist 91b80b70e2 Update HTML5 tags
From https://github.com/htacg/tidy-html5/blob/master/src/tags.c#L296
2015-08-19 23:12:31 +02:00
Jeremy Benoist 8667dae74b Merge pull request #4 from j0k3r/php53
Restore compatibility with PHP 5.3
2015-06-11 09:39:26 +02:00
Jeremy Benoist eecae93161 Try hhvm instead of nightly
HHVM nightly is no longer supported on Ubuntu Precise.
See https://github.com/travis-ci/travis-ci/issues/3788 and https://github.com/facebook/hhvm/issues/5220
2015-06-10 09:33:25 +02:00
Jeremy Benoist 814c6e4730 Restore compatibility with PHP 5.3 2015-06-10 09:33:21 +02:00
11 changed files with 901 additions and 363 deletions
+9
View File
@@ -0,0 +1,9 @@
/.editorconfig export-ignore
/.gitattributes export-ignore
/.gitignore export-ignore
/.scrutinizer.yml export-ignore
/.travis.yml export-ignore
/.php_cs export-ignore
/README.md export-ignore
/phpunit.xml.dist export-ignore
/tests export-ignore
+1
View File
@@ -1,3 +1,4 @@
vendor/
coverage/
composer.lock
.php_cs.cache
+20
View File
@@ -0,0 +1,20 @@
<?php
return Symfony\CS\Config\Config::create()
->setUsingCache(true)
->level(Symfony\CS\FixerInterface::SYMFONY_LEVEL)
// use default SYMFONY_LEVEL and extra fixers:
->fixers(array(
'concat_with_spaces',
'ordered_use',
'phpdoc_order',
'strict',
'strict_param',
'long_array_syntax',
))
->finder(
Symfony\CS\Finder\DefaultFinder::create()
->in(__DIR__)
->exclude(array('vendor'))
)
;
+1 -2
View File
@@ -1,3 +1,2 @@
tools:
external_code_coverage:
timeout: 600
external_code_coverage: false
+25 -9
View File
@@ -1,22 +1,38 @@
language: php
php:
- 5.3.3
- 5.3
- 5.4
- 5.5
- 5.6
- nightly
- hhvm-nightly
- 7.0
- hhvm
# run build against nightly but allow them to fail
matrix:
include:
- php: 7.0
env: CS_FIXER=run
fast_finish: true
allow_failures:
- php: nightly
- php: hhvm-nightly
- php: hhvm
# faster builds on new travis setup not using sudo
sudo: false
# cache vendor dirs
cache:
directories:
- vendor
- $HOME/.composer/cache
before_install:
- if [ -n "$GH_TOKEN" ]; then composer config github-oauth.github.com ${GH_TOKEN}; fi;
# disable TLS for composer because openssl is disabled for PHP 5.3.3 on travis
# see: https://blog.travis-ci.com/upcoming_ubuntu_11_10_migration/
- if [[ $TRAVIS_PHP_VERSION = 5.3.3 ]]; then composer config -g -- disable-tls true; fi;
- if [[ $TRAVIS_PHP_VERSION = 5.3.3 ]]; then composer config -g -- secure-http false; fi;
install:
- composer self-update
@@ -24,9 +40,9 @@ before_script:
- composer install --prefer-dist --no-interaction
script:
- phpunit --coverage-clover=coverage.clover
- mkdir -p build/logs
- phpunit -v --coverage-clover build/logs/clover.xml
- if [ "$CS_FIXER" = "run" ]; then php vendor/bin/php-cs-fixer fix --verbose --dry-run ; fi;
after_script:
- |
wget https://scrutinizer-ci.com/ocular.phar
php ocular.phar code-coverage:upload --format=php-clover coverage.clover
- php vendor/bin/coveralls -v
+28 -3
View File
@@ -1,13 +1,13 @@
# Readability
[![Build Status](https://travis-ci.org/j0k3r/php-readability.svg?branch=master)](https://travis-ci.org/j0k3r/php-readability)
[![Code Coverage](https://scrutinizer-ci.com/g/j0k3r/php-readability/badges/coverage.png?b=master)](https://scrutinizer-ci.com/g/j0k3r/php-readability/?branch=master)
[![Coverage Status](https://coveralls.io/repos/j0k3r/php-readability/badge.svg?branch=master&service=github)](https://coveralls.io/github/j0k3r/php-readability/?branch=master)
This is an extract of the Readability class from the [full-text-rss](https://github.com/Dither/full-text-rss) fork. It kind be defined as a better version of the original [php-readability](http://code.fivefilters.org/php-readability).
This is an extract of the Readability class from this [full-text-rss](https://github.com/Dither/full-text-rss) fork. It can be defined as a better version of the original [php-readability](https://bitbucket.org/fivefilters/php-readability/overview).
## Differences
The default php-readability lib is really old and needs to be improved. I found a great fork of [full-text-rss](http://fivefilters.org/content-only/) from @Dither which improve the Readability class.
The default php-readability lib is really old and needs to be improved. I found a great fork of full-text-rss from [@Dither](https://github.com/Dither/full-text-rss) which improve the Readability class.
- I've extracted the class from its fork to be able to use it out of the box
- I've added some simple tests
@@ -15,6 +15,12 @@ The default php-readability lib is really old and needs to be improved. I found
**But** the code is still really hard to understand / read ...
## Requirements
By default, this lib will use the [Tidy extension](https://github.com/htacg/tidy-html5) if it's available. Tidy is only used to cleanup the given HTML and avoid problems with bad HTML structure, etc ..
Since Composer doesn't support suggestion on PHP extension, I write this suggestion here.
## Usage
```php
@@ -26,6 +32,8 @@ $url = 'http://www.medialens.org/index.php/alerts/alert-archive/alerts-2013/729-
$html = file_get_contents($url);
$readability = new Readability($html, $url);
// or without Tidy
// $readability = new Readability($html, $url, 'libxml', false);
$result = $readability->init();
if ($result) {
@@ -37,3 +45,20 @@ if ($result) {
echo 'Looks like we couldn\'t find the content. :(';
}
```
If you want to debug it, or check what's going on, you can inject a logger (which must follow `Psr\Log\LoggerInterface`, Monolog for example):
```php
use Readability\Readability;
use Monolog\Logger;
use Monolog\Handler\StreamHandler;
$url = 'http://www.medialens.org/index.php/alerts/alert-archive/alerts-2013/729-thatcher.html';
$html = file_get_contents($url);
$logger = new Logger('readability');
$logger->pushHandler(new StreamHandler('path/to/your.log', Logger::DEBUG));
$readability = new Readability($html, $url);
$readability->setLogger($logger);
```
+6 -2
View File
@@ -24,8 +24,12 @@
"role": "Developer (original JS version)"
}],
"require": {
"php": ">=5.4",
"ext-tidy": ">=1.2"
"php": ">=5.3.3",
"monolog/monolog": "^1.13.1"
},
"require-dev": {
"satooshi/php-coveralls": "~0.6",
"friendsofphp/php-cs-fixer": "*"
},
"autoload": {
"psr-4": { "Readability\\": "src/" }
+3 -3
View File
@@ -26,7 +26,7 @@
</whitelist>
</filter>
<logging>
<log type="coverage-html" target="coverage" title="FullText" charset="UTF-8" yui="true" highlight="true" lowUpperBound="35" highLowerBound="70"/>
</logging>
<!-- <logging>
<log type="coverage-html" target="coverage" title="Readability" charset="UTF-8" yui="true" highlight="true" lowUpperBound="35" highLowerBound="70"/>
</logging> -->
</phpunit>
+16 -9
View File
@@ -45,16 +45,19 @@ class JSLikeHTMLElement extends \DOMElement
*/
public function __set($name, $value)
{
if ($name == 'innerHTML') {
if ($name === 'innerHTML') {
// first, empty the element
for ($x = $this->childNodes->length - 1; $x >= 0; $x--) {
for ($x = $this->childNodes->length - 1; $x >= 0; --$x) {
$this->removeChild($this->childNodes->item($x));
}
// $value holds our new inner HTML
if ($value != '') {
if ($value !== '') {
$f = $this->ownerDocument->createDocumentFragment();
// appendXML() expects well-formed markup (XHTML)
$result = @$f->appendXML($value); // @ to suppress PHP warnings
// @ to suppress PHP warnings
$result = @$f->appendXML($value);
if ($result) {
if ($f->hasChildNodes()) {
$this->appendChild($f);
@@ -63,14 +66,17 @@ class JSLikeHTMLElement extends \DOMElement
// $value is probably ill-formed
$f = new \DOMDocument();
$value = mb_convert_encoding($value, 'HTML-ENTITIES', 'UTF-8');
// Using <htmlfragment> will generate a warning, but so will bad HTML
// (and by this point, bad HTML is what we've got).
// We use it (and suppress the warning) because an HTML fragment will
// be wrapped around <html><body> tags which we don't really want to keep.
// Note: despite the warning, if loadHTML succeeds it will return true.
$result = @$f->loadHTML('<htmlfragment>'.$value.'</htmlfragment>');
$result = @$f->loadHTML('<htmlfragment>' . $value . '</htmlfragment>');
if ($result) {
$import = $f->getElementsByTagName('htmlfragment')->item(0);
foreach ($import->childNodes as $child) {
$importedNode = $this->ownerDocument->importNode($child, true);
$this->appendChild($importedNode);
@@ -83,7 +89,7 @@ class JSLikeHTMLElement extends \DOMElement
}
} else {
$trace = debug_backtrace();
trigger_error('Undefined property via __set(): '.$name.' in '.$trace[0]['file'].' on line '.$trace[0]['line'], E_USER_NOTICE);
trigger_error('Undefined property via __set(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], E_USER_NOTICE);
}
}
@@ -96,8 +102,9 @@ class JSLikeHTMLElement extends \DOMElement
*/
public function __get($name)
{
if ($name == 'innerHTML') {
if ($name === 'innerHTML') {
$inner = '';
foreach ($this->childNodes as $child) {
$inner .= $this->ownerDocument->saveXML($child);
}
@@ -106,13 +113,13 @@ class JSLikeHTMLElement extends \DOMElement
}
$trace = debug_backtrace();
trigger_error('Undefined property via __get(): '.$name.' in '.$trace[0]['file'].' on line '.$trace[0]['line'], E_USER_NOTICE);
trigger_error('Undefined property via __get(): ' . $name . ' in ' . $trace[0]['file'] . ' on line ' . $trace[0]['line'], E_USER_NOTICE);
return;
}
public function __toString()
{
return '['.$this->tagName.']';
return '[' . $this->tagName . ']';
}
}
+499 -290
View File
File diff suppressed because it is too large Load Diff
+293 -45
View File
@@ -2,99 +2,135 @@
namespace Tests\Readability;
use Monolog\Handler\TestHandler;
use Monolog\Logger;
use Readability\Readability;
class ReadabilityTested extends Readability
{
public function getDebugText()
{
return $this->debugText;
}
public function getDomainRegexp()
{
return $this->domainRegExp;
}
}
class ReadabilityTest extends \PHPUnit_Framework_TestCase
{
public $logHandler;
public $logger;
private function getReadability($html, $url = null, $parser = 'libxml', $useTidy = true)
{
$readability = new Readability($html, $url, $parser, $useTidy);
$this->logHandler = new TestHandler();
$this->logger = new Logger('test', array($this->logHandler));
$readability->setLogger($this->logger);
return $readability;
}
/**
* @requires extension tidy
*/
public function testConstructDefault()
{
$readability = new ReadabilityTested('');
$readability = $this->getReadability('');
$this->assertNull($readability->url);
$this->assertContains('Parsing URL', $readability->getDebugText());
$this->assertContains('Tidying document', $readability->getDebugText());
$this->assertNull($readability->getDomainRegexp());
$this->assertInstanceOf('DomDocument', $readability->dom);
}
/**
* @requires extension tidy
*/
public function testConstructSimple()
{
$readability = new ReadabilityTested('<html/>', 'http://0.0.0.0');
$readability = $this->getReadability('<html/>', 'http://0.0.0.0');
$this->assertEquals('http://0.0.0.0', $readability->url);
$this->assertContains('Parsing URL', $readability->getDebugText());
$this->assertContains('Tidying document', $readability->getDebugText());
$this->assertEquals('/0\.0\.0\.0/', $readability->getDomainRegexp());
$this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertEquals('<html/>', $readability->original_html);
$this->assertTrue($readability->tidied);
}
public function testConstructDefaultWithoutTidy()
{
$readability = $this->getReadability('', null, 'libxml', false);
$this->assertNull($readability->url);
$this->assertEquals('', $readability->original_html);
$this->assertFalse($readability->tidied);
$this->assertInstanceOf('DomDocument', $readability->dom);
}
public function testConstructSimpleWithoutTidy()
{
$readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'libxml', false);
$this->assertEquals('http://0.0.0.0', $readability->url);
$this->assertInstanceOf('DomDocument', $readability->dom);
$this->assertEquals('<html/>', $readability->original_html);
$this->assertFalse($readability->tidied);
}
public function testInitNoContent()
{
$readability = new ReadabilityTested('<html/>', 'http://0.0.0.0');
$readability = $this->getReadability('<html/>', 'http://0.0.0.0');
$res = $readability->init();
$this->assertFalse($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('Sorry, Readability was unable to parse this page for content.', $readability->getContent()->innerHTML);
}
public function testInitP()
{
$readability = new ReadabilityTested(str_repeat('<p>This is the awesome content :)</p>', 7), 'http://0.0.0.0');
$readability = $this->getReadability(str_repeat('<p>This is the awesome content :)</p>', 7), 'http://0.0.0.0');
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML);
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is the awesome content :)', $readability->getContent()->innerHTML);
}
public function testInitDivP()
{
$readability = new ReadabilityTested('<div>'.str_repeat('<p>This is the awesome content :)</p>', 7).'</div>', 'http://0.0.0.0');
$readability = $this->getReadability('<div>' . str_repeat('<p>This is the awesome content :)</p>', 7) . '</div>', 'http://0.0.0.0');
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML);
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is the awesome content :)', $readability->getContent()->innerHTML);
}
public function testInitDiv()
{
$readability = new ReadabilityTested('<div>'.str_repeat('This is the awesome content :)', 7).'</div>', 'http://0.0.0.0');
$readability = $this->getReadability('<div>' . str_repeat('This is the awesome content :)', 7) . '</div>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML);
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is the awesome content :)', $readability->getContent()->innerHTML);
}
public function testWithFootnotes()
{
$readability = new ReadabilityTested('<div>'.str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7).'</div>', 'http://0.0.0.0');
$readability = $this->getReadability('<div>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '</div>', 'http://0.0.0.0');
$readability->debug = true;
$readability->convertLinksToFootnotes = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML);
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertContains('readabilityFootnoteLink', $readability->getContent()->innerHTML);
$this->assertContains('readabilityLink-3', $readability->getContent()->innerHTML);
@@ -102,118 +138,330 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
public function testStandardClean()
{
$readability = new ReadabilityTested('<div><h2>Title</h2>'.str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7).'<a href="#nofollow" rel="nofollow">will be removed</a></div>', 'http://0.0.0.0');
$readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<a href="#nofollow" rel="nofollow">will NOT be removed</a></div>', 'http://0.0.0.0');
$readability->debug = true;
$readability->lightClean = false;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML);
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertNotContains('will be removed', $readability->getContent()->innerHTML);
$this->assertContains('will NOT be removed', $readability->getContent()->innerHTML);
$this->assertNotContains('<h2>', $readability->getContent()->innerHTML);
}
public function testWithIframe()
{
$readability = new ReadabilityTested('<div><h2>Title</h2>'.str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7).'<p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe><iframe>http://soundcloud.com/test</iframe></p></div>', 'http://0.0.0.0');
$readability = $this->getReadability('<div><h2>Title</h2>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe><iframe>http://soundcloud.com/test</iframe></p></div>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<div readability=', $readability->getContent()->innerHTML);
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertContains('nofollow', $readability->getContent()->innerHTML);
}
public function testWithArticle()
{
$readability = new ReadabilityTested('<article><p>'.str_repeat('This is an awesome text with some links, here there are: the awesome', 20).'</p><p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe></p></article>', 'http://0.0.0.0');
$readability = $this->getReadability('<article><p>' . str_repeat('This is an awesome text with some links, here there are: the awesome', 20) . '</p><p>This is an awesome text with some links, here there are <iframe src="http://youtube.com/test" href="#nofollow" rel="nofollow"></iframe></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->innerHTML);
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertContains('nofollow', $readability->getContent()->innerHTML);
}
public function testWithAside()
{
$readability = new ReadabilityTested('<article>'.str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7).'<footer><aside>'.str_repeat('<p>This is an awesome text with some links, here there are</p>', 8).'</aside></footer></article>', 'http://0.0.0.0');
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<footer><aside>' . str_repeat('<p>This is an awesome text with some links, here there are</p>', 8) . '</aside></footer></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertContains('alt="article"', $readability->getContent()->innerHTML);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertNotContains('<aside>', $readability->getContent()->innerHTML);
$this->assertContains('<footer/>', $readability->getContent()->innerHTML);
$this->assertContains('<footer readability="4"/>', $readability->getContent()->innerHTML);
}
public function testWithClasses()
{
$readability = new ReadabilityTested('<article>'.str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7).'<div style="display:none">'.str_repeat('<p class="clock">This text should be removed</p>', 10).'</div></article>', 'http://0.0.0.0');
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->innerHTML);
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertNotContains('This text should be removed', $readability->getContent()->innerHTML);
}
public function testWithClassesWithoutLightClean()
{
$readability = $this->getReadability('<article>' . str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7) . '<div style="display:none">' . str_repeat('<p class="clock">This text should be removed</p>', 10) . '</div></article>', 'http://0.0.0.0');
$readability->debug = true;
$readability->lightClean = false;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('alt="article"', $readability->getContent()->innerHTML);
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertNotContains('This text should be removed', $readability->getContent()->innerHTML);
}
public function testWithTd()
{
$readability = new ReadabilityTested('<table><tr>'.str_repeat('<td><p>This is an awesome text with some links, here there are the awesome</td>', 7).'</tr></table>', 'http://0.0.0.0');
$readability = $this->getReadability('<table><tr>' . str_repeat('<td><p>This is an awesome text with some links, here there are the awesome</td>', 7) . '</tr></table>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertContains('alt="tr"', $readability->getContent()->innerHTML);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
}
public function testWithSameClasses()
{
$readability = new ReadabilityTested('<article class="awesomecontent">'.str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7).'<div class="awesomecontent">This text is also an awesome text and you should know that !</div></article>', 'http://0.0.0.0');
$readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<div class="awesomecontent">This text is also an awesome text and you should know that !</div></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertContains('alt="article"', $readability->getContent()->innerHTML);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
}
public function testWithScript()
{
$readability = new ReadabilityTested('<article class="awesomecontent">'.str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7).'<p><script>This text is also an awesome text and you should know that !</script></p></article>', 'http://0.0.0.0');
$readability = $this->getReadability('<article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p><script>This text is also an awesome text and you should know that !</script></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertContains('alt="article"', $readability->getContent()->innerHTML);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEmpty($readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
}
public function testTitle()
{
$readability = $this->getReadability('<title>this is my title</title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEquals('this is my title', $readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
}
public function testTitleWithDash()
{
$readability = $this->getReadability('<title> title2 - title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEquals('title2 - title3', $readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
}
public function testTitleWithDoubleDot()
{
$readability = $this->getReadability('<title> title2 : title3 </title><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEquals('title2 : title3', $readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
}
public function testTitleTooShortUseH1()
{
$readability = $this->getReadability('<title>too short</title><h1>this is my h1 title !</h1><article class="awesomecontent">' . str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7) . '<p></p></article>', 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertEquals('this is my h1 title !', $readability->getTitle()->innerHTML);
$this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
$this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
}
// public function testConstructParser()
// {
// $readability = new ReadabilityTested('<html/>', 'http://0.0.0.0', 'html5lib');
// $readability = $this->getReadability('<html/>', 'http://0.0.0.0', 'html5lib');
// $this->assertEquals('http://0.0.0.0', $readability->url);
// $this->assertContains('Parsing URL', $readability->getDebugText());
// $this->assertContains('Tidying document', $readability->getDebugText());
// $this->assertEquals('/0\.0\.0\.0/', $readability->getDomainRegexp());
// $this->assertInstanceOf('DomDocument', $readability->dom);
// }
// dummy function to be used to the next test
public function error2Exception($code, $string, $file, $line, $context)
{
throw new \Exception($string, $code);
}
public function testAutoClosingIframeNotThrowingException()
{
error_reporting(E_ALL | E_STRICT);
ini_set('display_errors', true);
set_error_handler(array($this, 'error2Exception'), E_ALL | E_STRICT);
$data = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="ru-RU" prefix="og: http://ogp.me/ns#">
<head profile="http://gmpg.org/xfn/11">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
</head>
<body class="single single-post postid-22030 single-format-standard">
<div id="wrapper">
<div id="content">
<div class="post-22030 post type-post status-publish format-standard has-post-thumbnail hentry category-video category-reviews tag-193" id="post-22030">
<h1>3D Touch &#8212; будущее мобильных игр</h1>
<div class="postdate">Автор: <strong>Сергей Пак</strong> | Просмотров: 1363 | Опубликовано: 14 сентября 2015 </div>
<div class="entry">
<p>Компания Apple представила новую технологию 3D Touch, которая является прямым потомком более ранней версии Force Touch &#8212; последняя, напомним, используется сейчас в трекпадах Macbook Pro и Macbook 2015. Теперь управлять устройством стало в разы проще, и Force Touch открывает перед пользователями новые возможности, но при этом 3D Touch &#8212; это про другое. Дело в том, что теперь и на мобильных устройствах интерфейс будет постепенно меняться, кардинальные перемены ждут мобильный гейминг, потому что здесь разработчики действительно могут разгуляться.<span id="more-22030"></span></p>
<p><iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p>
<p>Итак, просто представьте себе, что iPhone 6S &#8212; это, по большому счету, отличная игровая приставка, которую вы носите с собой, а еще она может выдавать невероятной красоты картинку. Но проблема заключается, пожалуй, в том, что управлять персонажем в играх довольно трудно &#8212; он неповоротлив, обладает заторможенной реакцией, а игровой клиент зачастую требует перегруза интерфейса для того, чтобы обеспечить максимально большое количество возможностей. Благодаря трехуровневому нажатию можно избавиться от лишних кнопок и обеспечить более качественный обзор местности, и при этом пользователь будет закрывать пальцами минимальное пространство.</p>
</div>
</div>
</div>
</div>
</body>
</html>';
$readability = $this->getReadability($data, 'http://iosgames.ru/?p=22030');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
$this->assertContains('<iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"> </iframe>', $readability->getContent()->innerHTML);
$this->assertContains('3D Touch', $readability->getTitle()->innerHTML);
}
/**
* This should generate an Exception "DOMElement::setAttribute(): ID post-60 already defined".
*/
public function testAppendIdAlreadyHere()
{
$data = '<!DOCTYPE html>
<html lang="fr">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, user-scalable=yes, initial-scale=1.0">
</head>
<body>
<div class="container">
<header class="header sml-text-center med-text-left" role="banner">
<h1 class="no-margin"><a class="maintitle" href="https://0.0.0.0" title="Bloc-notes">Bloc-notes</a></h1>
<h2 class="h5 no-margin"></h2>
</header>
<nav class="nav" role="navigation">
<div class="responsive-menu">
<label for="menu">Menu</label>
<input type="checkbox" id="menu">
</div>
</nav>
<article class="article" role="article" id="post-60">
<section>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are<br/>
This is an awesome text with some links, here there are
</section>
<footer>
<small>
Classé dans : <a class="noactive" title="Services réseaux">Services réseaux</a>
</small>
</footer>
</article>
</div>
</body>
</html>';
$readability = $this->getReadability($data, 'http://0.0.0.0');
$readability->debug = true;
$res = $readability->init();
$this->assertTrue($res);
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
$this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
}
public function testPostFilters()
{
$readability = $this->getReadability('<div>' . str_repeat('<p>This <b>is</b> the awesome content :)</p>', 7) . '</div>', 'http://0.0.0.0');
$readability->addPostFilter('!<strong[^>]*>(.*?)</strong>!is', '');
$res = $readability->init();
$this->assertTrue($res);
$this->assertContains('This the awesome content :)', $readability->getContent()->innerHTML);
}
public function testPreFilters()
{
$this->markTestSkipped('Won\'t work until loadHtml() is moved in init() instead of __construct()');
$readability = $this->getReadability('<div>' . str_repeat('<p>This <b>is</b> the awesome and WONDERFUL content :)</p>', 7) . '</div>', 'http://0.0.0.0');
$readability->addPreFilter('!<b[^>]*>(.*?)</b>!is', '');
$res = $readability->init();
$this->assertTrue($res);
$this->assertContains('This the awesome and WONDERFUL content :)', $readability->getContent()->innerHTML);
}
}