Merge pull request #10 from j0k3r/remove-tidy-constraint

Remove tidy from requirement
Add Tidy requirement in README
2026-09-27 06:26:17 +00:00 · 2015-11-10 09:55:29 +01:00 · 2015-11-09 20:25:39 +01:00 · 2015-11-09 19:59:18 +01:00 · 2015-11-09 19:44:11 +01:00 · 2015-11-09 11:12:07 +01:00
6 changed files with 389 additions and 69 deletions
@@ -0,0 +1,8 @@
 /.editorconfig export-ignore
 /.gitattributes export-ignore
 /.gitignore export-ignore
 /.scrutinizer.yml export-ignore
 /.travis.yml export-ignore
 /README.md export-ignore
 /phpunit.xml.dist export-ignore
 /tests export-ignore
@@ -3,11 +3,11 @@
 [![Build Status](https://travis-ci.org/j0k3r/php-readability.svg?branch=master)](https://travis-ci.org/j0k3r/php-readability)
 [![Code Coverage](https://scrutinizer-ci.com/g/j0k3r/php-readability/badges/coverage.png?b=master)](https://scrutinizer-ci.com/g/j0k3r/php-readability/?branch=master)
-This is an extract of the Readability class from the [full-text-rss](https://github.com/Dither/full-text-rss) fork. It kind be defined as a better version of the original [php-readability](http://code.fivefilters.org/php-readability).
+This is an extract of the Readability class from this [full-text-rss](https://github.com/Dither/full-text-rss) fork. It can be defined as a better version of the original [php-readability](https://bitbucket.org/fivefilters/php-readability/overview).
 ## Differences
-The default php-readability lib is really old and needs to be improved. I found a great fork of [full-text-rss](http://fivefilters.org/content-only/) from @Dither which improve the Readability class.
+The default php-readability lib is really old and needs to be improved. I found a great fork of full-text-rss from [@Dither](https://github.com/Dither/full-text-rss) which improve the Readability class.
 - I've extracted the class from its fork to be able to use it out of the box
 - I've added some simple tests
@@ -15,6 +15,12 @@ The default php-readability lib is really old and needs to be improved. I found
 **But** the code is still really hard to understand / read ...
 ## Requirements
 By default, this lib will use the [Tidy extension](https://github.com/htacg/tidy-html5) if it's available. Tidy is only used to cleanup the given HTML and avoid problems with bad HTML structure, etc ..
 Since Composer doesn't support suggestion on PHP extension, I write this suggestion here.
 ## Usage
 ```php
@@ -26,6 +32,8 @@ $url = 'http://www.medialens.org/index.php/alerts/alert-archive/alerts-2013/729-
 $html = file_get_contents($url);
 $readability = new Readability($html, $url);
 // or without Tidy
 // $readability = new Readability($html, $url, 'libxml', false);
 $result = $readability->init();
 if ($result) {
@@ -24,8 +24,7 @@
        "role": "Developer (original JS version)"
    }],
    "require": {
-        "php": ">=5.3.3",
+        "php": ">=5.3.3"
        "ext-tidy": ">=1.2"
    },
    "autoload": {
        "psr-4": { "Readability\\": "src/" }
@@ -47,12 +47,14 @@ class JSLikeHTMLElement extends \DOMElement
    {
        if ($name == 'innerHTML') {
            // first, empty the element
-            for ($x = $this->childNodes->length - 1; $x >= 0; $x--) {
+            for ($x = $this->childNodes->length - 1; $x >= 0; --$x) {
                $this->removeChild($this->childNodes->item($x));
            }
            // $value holds our new inner HTML
            if ($value != '') {
                $f = $this->ownerDocument->createDocumentFragment();
                // appendXML() expects well-formed markup (XHTML)
                $result = @$f->appendXML($value); // @ to suppress PHP warnings
                if ($result) {
@@ -63,12 +65,14 @@ class JSLikeHTMLElement extends \DOMElement
                    // $value is probably ill-formed
                    $f = new \DOMDocument();
                    $value = mb_convert_encoding($value, 'HTML-ENTITIES', 'UTF-8');
                    // Using <htmlfragment> will generate a warning, but so will bad HTML
                    // (and by this point, bad HTML is what we've got).
                    // We use it (and suppress the warning) because an HTML fragment will
                    // be wrapped around <html><body> tags which we don't really want to keep.
                    // Note: despite the warning, if loadHTML succeeds it will return true.
                    $result = @$f->loadHTML('<htmlfragment>'.$value.'</htmlfragment>');
                    if ($result) {
                        $import = $f->getElementsByTagName('htmlfragment')->item(0);
                        foreach ($import->childNodes as $child) {
@@ -63,15 +63,16 @@ class Readability
    protected $bodyCache = null; // Cache the body HTML in case we need to re-use it later
    protected $flags = 7; // 1 | 2 | 4;   // Start with all processing flags set.
    protected $success = false; // indicates whether we were able to extract or not
    /**
     * All of the regular expressions in use within readability.
     * Defined up here so we don't instantiate them repeatedly in loops.
     */
    public $regexps = array(
-        'unlikelyCandidates' => '/display\s*:\s*none|ignore|\binfo|annoy|clock|date|time|author|intro|links|hidd?e|about|archive|\bprint|bookmark|tags|share|search|social|robot|published|combx|comment|mast(?:head)|subscri|community|category|disqus|extra|head(?:er|note)|floor|foot(?:er|note)|menu|tool|function|nav|remark|rss|shoutbox|tool|widget|meta|banner|sponsor|adsense|inner-?ad|ad-|sponsor|\badv\b|\bads\b|agr?egate?|pager|sidebar|popup|tweet|twitter/i',
+        'unlikelyCandidates' => '/display\s*:\s*none|ignore|\binfo|annoy|clock|date|time|author|intro|links|hidd?e|about|archive|\bprint|bookmark|tags|share|search|social|robot|published|combx|comment|mast(?:head)|subscri|community|category|disqus|extra|head|head(?:er|note)|floor|foot(?:er|note)|menu|tool|function|nav|remark|rss|shoutbox|tool|widget|meta|banner|sponsor|adsense|inner-?ad|ad-|sponsor|\badv\b|\bads\b|agr?egate?|pager|sidebar|popup|tweet|twitter/i',
        'okMaybeItsACandidate' => '/article\b|contain|\bcontent|column|general|detail|shadow|lightbox|blog|body|entry|main|page/i',
        'positive' => '/read|full|article|body|\bcontent|contain|entry|main|markdown|page|attach|pagination|post|text|blog|story/i',
-        'negative' => '/bottom|stat|info|discuss|e[\-]?mail|comment|reply|log.{2}(n|ed)|sign|single|combx|com-|contact|_nav|link|media|\bout|promo|\bad-|related|scroll|shoutbox|sidebar|sponsor|shopping|teaser/i',
+        'negative' => '/bottom|stat|info|discuss|e[\-]?mail|comment|reply|log.{2}(n|ed)|sign|single|combx|com-|contact|_nav|link|media|\bout|promo|\bad-|related|scroll|shoutbox|sidebar|sponsor|shopping|teaser|recommend/i',
        'divToPElements' => '/<(?:blockquote|code|div|article|footer|aside|img|p|pre|dl|ol|ul)/mi',
        'killBreaks' => '/(<br\s*\/?>([ \r\n\s]|&nbsp;?)*)+/',
        'media' => '!//(?:[^\.\?/]+\.)?(?:youtu(?:be)?|soundcloud|dailymotion|vimeo|pornhub|xvideos|twitvid|rutube|viddler)\.(?:com|be|org|net)/!i',
@@ -122,6 +123,7 @@ class Readability
        '!</code>\s*</pre>!is' => '</pre>',
        '!<[hb]r>!is' => '<\\1 />',
    );
    // flags
    const FLAG_STRIP_UNLIKELYS = 1;
    const FLAG_WEIGHT_ATTRIBUTES = 2;
@@ -137,6 +139,7 @@ class Readability
    const MIN_ARTICLE_LENGTH = 200;
    const MIN_NODE_LENGTH = 80;
    const MAX_LINK_DENSITY = 0.25;
    /**
     * Create instance of Readability.
     *
@@ -183,15 +186,16 @@ class Readability
                $this->original_html = $html;
                $this->tidied = true;
                $html = $tidy->value;
                $html = preg_replace('/<html[^>]+>/i', '<html>', $html);
                $html = preg_replace('/[\r\n]+/is', "\n", $html);
            }
            unset($tidy);
        }
        $html = mb_convert_encoding($html, 'HTML-ENTITIES', 'UTF-8');
        if (!($parser == 'html5lib' && ($this->dom = \HTML5_Parser::parse($html)))) {
            libxml_use_internal_errors(true);
            $this->dom = new \DOMDocument();
            $this->dom->preserveWhiteSpace = false;
@@ -206,6 +210,7 @@ class Readability
        $this->dom->registerNodeClass('DOMElement', 'Readability\JSLikeHTMLElement');
    }
    /**
     * Get article title element.
     *
@@ -215,6 +220,7 @@ class Readability
    {
        return $this->articleTitle;
    }
    /**
     * Get article content element.
     *
@@ -224,6 +230,7 @@ class Readability
    {
        return $this->articleContent;
    }
    /**
     * Add pre filter for raw input HTML processing.
     *
@@ -234,6 +241,7 @@ class Readability
    {
        $this->pre_filters[$filter] = $replacer;
    }
    /**
     * Add post filter for raw output HTML processing.
     *
@@ -244,6 +252,7 @@ class Readability
    {
        $this->post_filters[$filter] = $replacer;
    }
    /**
     * Runs readability.
     *
@@ -265,6 +274,7 @@ class Readability
        // Assume successful outcome
        $this->success = true;
        $bodyElems = $this->dom->getElementsByTagName('body');
        // WTF multiple body nodes?
        if ($this->bodyCache == null) {
            $this->bodyCache = '';
@@ -272,32 +282,40 @@ class Readability
                $this->bodyCache .= trim($bodyNode->innerHTML);
            }
        }
        if ($bodyElems->length > 0 && $this->body == null) {
            $this->body = $bodyElems->item(0);
        }
        $this->prepDocument();
        // Build readability's DOM tree.
        $overlay = $this->dom->createElement('div');
        $innerDiv = $this->dom->createElement('div');
-        $articleTitle   = $this->getArticleTitle();
+        $articleTitle = $this->getArticleTitle();
        $articleContent = $this->grabArticle();
        if (!$articleContent) {
            $this->success = false;
            $articleContent = $this->dom->createElement('div');
-            $articleContent->setAttribute('id', 'readability-content');
+            $articleContent->setAttribute('class', 'readability-content');
            $articleContent->innerHTML = '<p>Sorry, Readability was unable to parse this page for content.</p>';
        }
-        $overlay->setAttribute('id', 'readOverlay');
+
-        $innerDiv->setAttribute('id', 'readInner');
+        $overlay->setAttribute('class', 'readOverlay');
        $innerDiv->setAttribute('class', 'readInner');
        // Glue the structure of our document together.
        $innerDiv->appendChild($articleTitle);
        $innerDiv->appendChild($articleContent);
        $overlay->appendChild($innerDiv);
        // Clear the old HTML, insert the new content.
        $this->body->innerHTML = '';
        $this->body->appendChild($overlay);
        $this->body->removeAttribute('style');
        $this->postProcessContent($articleContent);
        // Set title and content instance variables.
        $this->articleTitle = $articleTitle;
        $this->articleContent = $articleContent;
@@ -305,6 +323,7 @@ class Readability
        return $this->success;
    }
    /**
     * Debug.
     */
@@ -325,6 +344,7 @@ class Readability
            syslog(6, $this->debugText); // 1 - error 6 - info
        }
    }
    /**
     * Run any post-process modifications to article content as necessary.
     *
@@ -336,6 +356,7 @@ class Readability
            $this->addFootnotes($articleContent);
        }
    }
    /**
     * Get the article title as an H1.
     *
@@ -345,10 +366,12 @@ class Readability
    {
        $curTitle = '';
        $origTitle = '';
        try {
            $curTitle = $origTitle = $this->getInnerText($this->dom->getElementsByTagName('title')->item(0));
        } catch (Exception $e) {
        }
        if (preg_match('/ [\|\-] /', $curTitle)) {
            $curTitle = preg_replace('/(.*)[\|\-] .*/i', '$1', $origTitle);
            if (count(explode(' ', $curTitle)) < 3) {
@@ -365,15 +388,18 @@ class Readability
                $curTitle = $this->getInnerText($hOnes->item(0));
            }
        }
        $curTitle = trim($curTitle);
        if (count(explode(' ', $curTitle)) <= 4) {
            $curTitle = $origTitle;
        }
        $articleTitle = $this->dom->createElement('h1');
        $articleTitle->innerHTML = $curTitle;
        return $articleTitle;
    }
    /**
     * Prepare the HTML document for readability to scrape it.
     * This includes things like stripping javascript, CSS, and handling terrible markup.
@@ -388,17 +414,21 @@ class Readability
            $this->body = $this->dom->createElement('body');
            $this->dom->documentElement->appendChild($this->body);
        }
-        $this->body->setAttribute('id', 'readabilityBody');
+
        $this->body->setAttribute('class', 'readabilityBody');
        // Remove all style tags in head.
        $styleTags = $this->dom->getElementsByTagName('style');
-        for ($i = $styleTags->length - 1; $i >= 0; $i--) {
+        for ($i = $styleTags->length - 1; $i >= 0; --$i) {
            $styleTags->item($i)->parentNode->removeChild($styleTags->item($i));
        }
        $linkTags = $this->dom->getElementsByTagName('link');
-        for ($i = $linkTags->length - 1; $i >= 0; $i--) {
+        for ($i = $linkTags->length - 1; $i >= 0; --$i) {
            $linkTags->item($i)->parentNode->removeChild($linkTags->item($i));
        }
    }
    /**
     * For easier reading, convert this document to have footnotes at the bottom rather than inline links.
     *
@@ -407,52 +437,61 @@ class Readability
    public function addFootnotes($articleContent)
    {
        $footnotesWrapper = $this->dom->createElement('footer');
-        $footnotesWrapper->setAttribute('id', 'readability-footnotes');
+        $footnotesWrapper->setAttribute('class', 'readability-footnotes');
        $footnotesWrapper->innerHTML = '<h3>References</h3>';
        $articleFootnotes = $this->dom->createElement('ol');
-        $articleFootnotes->setAttribute('id', 'readability-footnotes-list');
+        $articleFootnotes->setAttribute('class', 'readability-footnotes-list');
        $footnotesWrapper->appendChild($articleFootnotes);
        $articleLinks = $articleContent->getElementsByTagName('a');
        $linkCount = 0;
-        for ($i = 0; $i < $articleLinks->length; $i++) {
+
-            $articleLink  = $articleLinks->item($i);
+        for ($i = 0; $i < $articleLinks->length; ++$i) {
            $articleLink = $articleLinks->item($i);
            $footnoteLink = $articleLink->cloneNode(true);
            $refLink = $this->dom->createElement('a');
            $footnote = $this->dom->createElement('li');
-            $linkDomain   = @parse_url($footnoteLink->getAttribute('href'), PHP_URL_HOST);
+            $linkDomain = @parse_url($footnoteLink->getAttribute('href'), PHP_URL_HOST);
            if (!$linkDomain && isset($this->url)) {
                $linkDomain = @parse_url($this->url, PHP_URL_HOST);
            }
            $linkText = $this->getInnerText($articleLink);
            if ((strpos($articleLink->getAttribute('class'), 'readability-DoNotFootnote') !== false) || preg_match($this->regexps['skipFootnoteLink'], $linkText)) {
                continue;
            }
-            $linkCount++;
+
            ++$linkCount;
            // Add a superscript reference after the article link.
            $refLink->setAttribute('href', '#readabilityFootnoteLink-'.$linkCount);
            $refLink->innerHTML = '<small><sup>['.$linkCount.']</sup></small>';
            $refLink->setAttribute('class', 'readability-DoNotFootnote');
            $refLink->setAttribute('style', 'color: inherit;');
            if ($articleLink->parentNode->lastChild->isSameNode($articleLink)) {
                $articleLink->parentNode->appendChild($refLink);
            } else {
                $articleLink->parentNode->insertBefore($refLink, $articleLink->nextSibling);
            }
            $articleLink->setAttribute('style', 'color: inherit; text-decoration: none;');
            $articleLink->setAttribute('name', 'readabilityLink-'.$linkCount);
            $footnote->innerHTML = '<small><sup><a href="#readabilityLink-'.$linkCount.'" title="Jump to Link in Article">^</a></sup></small> ';
            $footnoteLink->innerHTML = ($footnoteLink->getAttribute('title') != '' ? $footnoteLink->getAttribute('title') : $linkText);
            $footnoteLink->setAttribute('name', 'readabilityFootnoteLink-'.$linkCount);
            $footnote->appendChild($footnoteLink);
            if ($linkDomain) {
                $footnote->innerHTML = $footnote->innerHTML.'<small> ('.$linkDomain.')</small>';
            }
            $articleFootnotes->appendChild($footnote);
        }
        if ($linkCount > 0) {
            $articleContent->appendChild($footnotesWrapper);
        }
    }
    /**
     * Prepare the article node for display. Clean out any inline styles,
     * iframes, forms, strip extraneous <p> tags, etc.
@@ -466,31 +505,29 @@ class Readability
        } else {
            $this->dbg('Standard clean enabled.');
        }
        $this->cleanStyles($articleContent);
        $this->killBreaks($articleContent);
        $xpath = new \DOMXPath($articleContent->ownerDocument);
        if ($this->revertForcedParagraphElements) {
            /*
             * Reverts P elements with class 'readability-styled' to text nodes:
             * which is what they were before.
             */
            $elems = $xpath->query('.//p[@data-readability-styled]', $articleContent);
-            for ($i = $elems->length - 1; $i >= 0; $i--) {
+            for ($i = $elems->length - 1; $i >= 0; --$i) {
                $e = $elems->item($i);
                $e->parentNode->replaceChild($articleContent->ownerDocument->createTextNode($e->textContent), $e);
            }
        }
        // Remove service data-candidate attribute.
        $elems = $xpath->query('.//*[@data-candidate]', $articleContent);
-        for ($i = $elems->length - 1; $i >= 0; $i--) {
+        for ($i = $elems->length - 1; $i >= 0; --$i) {
            $elems->item($i)->removeAttribute('data-candidate');
        }
-        // Remove unrelated links and other unneded stuff.
+
        // (not(*) and not(text()[normalize-space()])) or // What's wrong here?
        $elems = $xpath->query('.//a[@rel="nofollow"]', $articleContent);
        for ($i = $elems->length - 1; $i >= 0; $i--) {
            $elems->item($i)->parentNode->removeChild($elems->item($i));
        }
        // Clean out junk from the article content.
        $this->clean($articleContent, 'input');
        $this->clean($articleContent, 'button');
@@ -508,29 +545,35 @@ class Readability
        if ($h2s->length == 1 && mb_strlen($this->getInnerText($h2s->item(0), true, true)) < 100) {
            $this->clean($articleContent, 'h2');
        }
        $this->cleanHeaders($articleContent);
        // Do these last as the previous stuff may have removed junk that will affect these.
        $this->cleanConditionally($articleContent, 'form');
        $this->cleanConditionally($articleContent, 'table');
        $this->cleanConditionally($articleContent, 'ul');
        //if (!$this->lightClean)
        $this->cleanConditionally($articleContent, 'div');
        // Remove extra paragraphs.
        $articleParagraphs = $articleContent->getElementsByTagName('p');
-        for ($i = $articleParagraphs->length - 1; $i >= 0; $i--) {
+
        for ($i = $articleParagraphs->length - 1; $i >= 0; --$i) {
            $imgCount = $articleParagraphs->item($i)->getElementsByTagName('img')->length;
            $embedCount = $articleParagraphs->item($i)->getElementsByTagName('embed')->length;
            $objectCount = $articleParagraphs->item($i)->getElementsByTagName('object')->length;
            $videoCount = $articleParagraphs->item($i)->getElementsByTagName('video')->length;
            $audioCount = $articleParagraphs->item($i)->getElementsByTagName('audio')->length;
            $iframeCount = $articleParagraphs->item($i)->getElementsByTagName('iframe')->length;
            if ($iframeCount === 0 && $imgCount === 0 && $embedCount === 0 && $objectCount === 0 && $videoCount === 0 && $audioCount === 0 && mb_strlen(preg_replace('/\s+/is', '', $this->getInnerText($articleParagraphs->item($i), false, false))) === 0) {
                $articleParagraphs->item($i)->parentNode->removeChild($articleParagraphs->item($i));
            }
            // add extra text to iframe tag to avoid an auto-closing iframe and then break the html code
            if ($iframeCount) {
                $iframe = $articleParagraphs->item($i)->getElementsByTagName('iframe');
-                $iframe->item(0)->nodeValue = '&nbsp;';
+                $iframe->item(0)->nodeValue = ' ';
                $articleParagraphs->item($i)->parentNode->replaceChild($iframe->item(0), $articleParagraphs->item($i));
            }
@@ -547,6 +590,7 @@ class Readability
            }
        }
    }
    /**
     * Initialize a node with the readability object. Also checks the
     * className/id for special names to add to its score.
@@ -612,8 +656,10 @@ class Readability
                $readability->value -= 5;
                break;
        }
        $readability->value += $this->getWeight($node);
    }
    /**
     * grabArticle - Using a variety of metrics (content score, classname, element types), find the content that is
     * most likely to be the stuff a user wants to read. Then return it wrapped up in a div.
@@ -625,43 +671,51 @@ class Readability
        if (!$page) {
            $page = $this->dom;
        }
        $xpath = null;
        $nodesToScore = array();
        if ($page instanceof \DOMDocument && isset($page->documentElement)) {
            $xpath = new \DOMXPath($page);
        }
        $allElements = $page->getElementsByTagName('*');
-        for ($nodeIndex = 0; ($node = $allElements->item($nodeIndex)); $nodeIndex++) {
+
        for ($nodeIndex = 0; ($node = $allElements->item($nodeIndex)); ++$nodeIndex) {
            $tagName = $node->tagName;
            // Some well known site uses sections as paragraphs.
            if (strcasecmp($tagName, 'p') === 0 || strcasecmp($tagName, 'td') === 0 || strcasecmp($tagName, 'section') === 0) {
                $nodesToScore[] = $node;
            }
            // Turn divs into P tags where they have been used inappropriately
            //  (as in, where they contain no other block level elements).
            if (strcasecmp($tagName, 'div') === 0 || strcasecmp($tagName, 'article') === 0 || strcasecmp($tagName, 'section') === 0) {
                if (!preg_match($this->regexps['divToPElements'], $node->innerHTML)) {
                    //$this->dbg('Altering '.$node->getNodePath().' to p');
                    $newNode = $this->dom->createElement('p');
                    try {
                        $newNode->innerHTML = $node->innerHTML;
                        // It's easier to debug using original attributes.
                        //$newNode->setAttribute('class', $node->getAttribute('class'));
                        //$newNode->setAttribute('id', $node->getAttribute('id'));
                        $node = $node->parentNode->replaceChild($newNode, $node);
-                        $nodeIndex--;
+                        --$nodeIndex;
                        $nodesToScore[] = $newNode;
                    } catch (Exception $e) {
                        $this->dbg('Could not alter div/article to p, reverting back to div: '.$e->getMessage());
                    }
                } else {
                    // Will change these P elements back to text nodes after processing.
-                    for ($i = 0, $il = $node->childNodes->length; $i < $il; $i++) {
+                    for ($i = 0, $il = $node->childNodes->length; $i < $il; ++$i) {
                        $childNode = $node->childNodes->item($i);
                        if (is_object($childNode) && get_class($childNode) === 'DOMProcessingInstruction') { //executable tags (<?php or <?xml) warning
                            $childNode->parentNode->removeChild($childNode);
                            continue;
                        }
                        if ($childNode->nodeType == 3) { // XML_TEXT_NODE
                            //$this->dbg('replacing text node with a P tag with the same content.');
                            $p = $this->dom->createElement('p');
@@ -673,6 +727,7 @@ class Readability
                }
            }
        }
        /*
         * Loop through all paragraphs, and assign a score to them based on how content-y they look.
         * Then add their score to their parent node.
@@ -680,23 +735,27 @@ class Readability
         * A score is determined by things like number of commas, class names, etc.
         * Maybe eventually link density.
         */
-        for ($pt = 0, $scored = count($nodesToScore); $pt < $scored; $pt++) {
+        for ($pt = 0, $scored = count($nodesToScore); $pt < $scored; ++$pt) {
            $parentNode = $nodesToScore[$pt]->parentNode;
            // No parent node? Move on...
            if (!$parentNode) {
                continue;
            }
            $grandParentNode = ($parentNode->parentNode instanceof DOMElement) ? $parentNode->parentNode : null;
            $innerText = $this->getInnerText($nodesToScore[$pt]);
            // If this paragraph is less than MIN_PARAGRAPH_LENGTH (default:20) characters, don't even count it.
            if (mb_strlen($innerText) < self::MIN_PARAGRAPH_LENGTH) {
                continue;
            }
            // Initialize readability data for the parent.
            if (!$parentNode->hasAttribute('readability')) {
                $this->initializeNode($parentNode);
                $parentNode->setAttribute('data-candidate', 'true');
            }
            // Initialize readability data for the grandparent.
            if ($grandParentNode && !$grandParentNode->hasAttribute('readability') && isset($grandParentNode->tagName)) {
                $this->initializeNode($grandParentNode);
@@ -723,6 +782,7 @@ class Readability
            }
            $score = floor($score);
            $contentScore += max(min($score, 3), -3);/**/
            // Add the score to the parent. The grandparent gets half.
            $parentNode->getAttributeNode('readability')->value += $contentScore;
            if ($grandParentNode) {
@@ -735,7 +795,8 @@ class Readability
         */
        if ($this->flagIsActive(self::FLAG_STRIP_UNLIKELYS) && $xpath) {
            $candidates = $xpath->query('.//*[(self::footer and count(//footer)<2) or (self::aside and count(//aside)<2)]', $page->documentElement);
-            for ($node = null, $c = $candidates->length - 1; $c >= 0; $c--) {
+
            for ($node = null, $c = $candidates->length - 1; $c >= 0; --$c) {
                $node = $candidates->item($c);
                // node should be readable but not inside of an article otherwise it's probably non-readable block
                if ($node->hasAttribute('readability') && (int) $node->getAttributeNode('readability')->value < 40 && ($node->parentNode ? strcasecmp($node->parentNode->tagName, 'article') !== 0 : true)) {
@@ -743,8 +804,10 @@ class Readability
                    $node->parentNode->removeChild($node);
                }
            }
            $candidates = $xpath->query('.//*[not(self::body) and (@class or @id or @style) and ((number(@readability) < 40) or not(@readability))]', $page->documentElement);
-            for ($node = null, $c = $candidates->length - 1; $c >= 0; $c--) {
+
            for ($node = null, $c = $candidates->length - 1; $c >= 0; --$c) {
                $node = $candidates->item($c);
                $tagName = $node->tagName;
                /* Remove unlikely candidates */
@@ -756,11 +819,12 @@ class Readability
                ) {
                    $this->dbg('Removing unlikely candidate '.$node->getNodePath().' by "'.$unlikelyMatchString.'" with readability '.($node->hasAttribute('readability') ? (int) $node->getAttributeNode('readability')->value : 0));
                    $node->parentNode->removeChild($node);
-                    $nodeIndex--;
+                    --$nodeIndex;
                }
            }
            unset($candidates);
        }
        /*
         * After we've calculated scores, loop through all of the possible candidate nodes we found
         * and find the one with the highest score.
@@ -769,25 +833,30 @@ class Readability
        if ($xpath) {
            // Using array of DOMElements after deletion is a path to DOOMElement.
            $candidates = $xpath->query('.//*[@data-candidate]', $page->documentElement);
-            for ($c = $candidates->length - 1; $c >= 0; $c--) {
+
            for ($c = $candidates->length - 1; $c >= 0; --$c) {
                // Scale the final candidates score based on link density. Good content should have a
                // relatively small link density (5% or less) and be mostly unaffected by this operation.
                // If not for this we would have used XPath to find maximum @readability.
                $readability = $candidates->item($c)->getAttributeNode('readability');
                $readability->value = round($readability->value * (1 - $this->getLinkDensity($candidates->item($c))), 0, PHP_ROUND_HALF_UP);
                if (!$topCandidate || $readability->value > (int) $topCandidate->getAttribute('readability')) {
                    $this->dbg('Candidate: '.$candidates->item($c)->getNodePath().' ('.$candidates->item($c)->getAttribute('class').':'.$candidates->item($c)->getAttribute('id').') with score '.$readability->value);
                    $topCandidate = $candidates->item($c);
                }
            }
            unset($candidates);
        }
        /*
         * If we still have no top candidate, just use the body as a last resort.
         * We also have to copy the body node so it is something we can modify.
         */
        if ($topCandidate === null || strcasecmp($topCandidate->tagName, 'body') === 0) {
            $topCandidate = $this->dom->createElement('div');
            if ($page instanceof \DOMDocument) {
                if (!isset($page->documentElement)) {
                    // we don't have a body either? what a mess! :)
@@ -796,6 +865,7 @@ class Readability
                    $this->dbg('Setting body to a raw HTML of original page!');
                    $topCandidate->innerHTML = $page->documentElement->innerHTML;
                    $page->documentElement->innerHTML = '';
                    $this->reinitBody();
                    $page->documentElement->appendChild($topCandidate);
                }
            } else {
@@ -803,81 +873,102 @@ class Readability
                $page->innerHTML = '';
                $page->appendChild($topCandidate);
            }
            $this->initializeNode($topCandidate);
        }
        // Set table as the main node if resulted data is table element.
        $tagName = $topCandidate->tagName;
        if (strcasecmp($tagName, 'td') === 0 || strcasecmp($tagName, 'tr') === 0) {
            $up = $topCandidate;
            if ($up->parentNode instanceof DOMElement) {
                $up = $up->parentNode;
                if (strcasecmp($up->tagName, 'table') === 0) {
                    $topCandidate = $up;
                }
            }
        }
        $this->dbg('Top candidate: '.$topCandidate->getNodePath());
        /*
         * Now that we have the top candidate, look through its siblings for content that might also be related.
         * Things like preambles, content split by ads that we removed, etc.
         */
        $articleContent = $this->dom->createElement('div');
-        $articleContent->setAttribute('id', 'readability-content');
+        $articleContent->setAttribute('class', 'readability-content');
        $siblingScoreThreshold = max(10, ((int) $topCandidate->getAttribute('readability')) * 0.2);
        $siblingNodes = $topCandidate->parentNode->childNodes;
        if (!isset($siblingNodes)) {
            $siblingNodes = new stdClass();
            $siblingNodes->length = 0;
        }
-        for ($s = 0, $sl = $siblingNodes->length; $s < $sl; $s++) {
+
        for ($s = 0, $sl = $siblingNodes->length; $s < $sl; ++$s) {
            $siblingNode = $siblingNodes->item($s);
            $siblingNodeName = $siblingNode->nodeName;
            $append = false;
            $this->dbg('Looking at sibling node: '.$siblingNode->getNodePath().(($siblingNode->nodeType === XML_ELEMENT_NODE && $siblingNode->hasAttribute('readability')) ? (' with score '.$siblingNode->getAttribute('readability')) : ''));
            //$this->dbg('Sibling has score ' . ($siblingNode->readability ? siblingNode.readability.contentScore : 'Unknown'));
            if ($siblingNode->isSameNode($topCandidate)) {
                $append = true;
            }
            $contentBonus = 0;
            // Give a bonus if sibling nodes and top candidates have the same classname.
            if ($siblingNode->nodeType === XML_ELEMENT_NODE && $siblingNode->getAttribute('class') == $topCandidate->getAttribute('class') && $topCandidate->getAttribute('class') != '') {
                $contentBonus += ((int) $topCandidate->getAttribute('readability')) * 0.2;
            }
            if ($siblingNode->nodeType === XML_ELEMENT_NODE && $siblingNode->hasAttribute('readability') && (((int) $siblingNode->getAttribute('readability')) + $contentBonus) >= $siblingScoreThreshold) {
                $append = true;
            }
            if (strcasecmp($siblingNodeName, 'p') === 0) {
                $linkDensity = $this->getLinkDensity($siblingNode);
                $nodeContent = $this->getInnerText($siblingNode, true, true);
-                $nodeLength  = mb_strlen($nodeContent);
+                $nodeLength = mb_strlen($nodeContent);
                if ($nodeLength > self::MIN_NODE_LENGTH && $linkDensity < self::MAX_LINK_DENSITY) {
                    $append = true;
                } elseif ($nodeLength < self::MIN_NODE_LENGTH && $linkDensity === 0 && preg_match('/\.( |$)/', $nodeContent)) {
                    $append = true;
                }
            }
            if ($append) {
                $this->dbg('Appending node: '.$siblingNode->getNodePath());
                $nodeToAppend = null;
                if (strcasecmp($siblingNodeName, 'div') !== 0 && strcasecmp($siblingNodeName, 'p') !== 0) {
                    /* We have a node that isn't a common block level element, like a form or td tag. Turn it into a div so it doesn't get filtered out later by accident. */
                    $this->dbg('Altering siblingNode '.$siblingNodeName.' to div.');
                    $nodeToAppend = $this->dom->createElement('div');
                    try {
-                        $nodeToAppend->setAttribute('id', $siblingNode->getAttribute('id'));
+                        if ($siblingNode->getAttribute('id')) {
                            $nodeToAppend->setAttribute('id', $siblingNode->getAttribute('id'));
                        }
                        $nodeToAppend->setAttribute('alt', $siblingNodeName);
                        $nodeToAppend->innerHTML = $siblingNode->innerHTML;
                    } catch (Exception $e) {
                        $this->dbg('Could not alter siblingNode '.$siblingNodeName.' to div, reverting to original.');
                        $nodeToAppend = $siblingNode;
-                        $s--;
+                        --$s;
-                        $sl--;
+                        --$sl;
                    }
                } else {
                    $nodeToAppend = $siblingNode;
-                    $s--;
+                    --$s;
-                    $sl--;
+                    --$sl;
                }
                // To ensure a node does not interfere with readability styles, remove its classnames & ids.
                // Now done via RegExp post_filter.
                //$nodeToAppend->removeAttribute('class');
@@ -886,9 +977,12 @@ class Readability
                $articleContent->appendChild($nodeToAppend);
            }
        }
        unset($xpath);
        // So we have all of the content that we need. Now we clean it up for presentation.
        $this->prepArticle($articleContent);
        /*
         * Now that we've gone through the full algorithm, check to see if we got any meaningful content.
         * If we didn't, we may need to re-run grabArticle with different flags set. This gives us a higher
@@ -896,10 +990,8 @@ class Readability
         * finding the -right- content.
         */
        if (mb_strlen($this->getInnerText($articleContent, false)) < self::MIN_ARTICLE_LENGTH) {
-            if (!$this->body->hasChildNodes()) {
+            $this->reinitBody();
-                $this->body = $this->dom->createElement('body');
+
            }
            $this->body->innerHTML = $this->bodyCache;
            if ($this->flagIsActive(self::FLAG_STRIP_UNLIKELYS)) {
                $this->removeFlag(self::FLAG_STRIP_UNLIKELYS);
                $this->dbg('...content is shorter than '.self::MIN_ARTICLE_LENGTH." letters, trying not to strip unlikely content.\n");
@@ -915,13 +1007,14 @@ class Readability
                $this->dbg('...content is shorter than '.self::MIN_ARTICLE_LENGTH." letters, trying not to clean at all.\n");
                return $this->grabArticle($this->body);
            } else {
                return false;
            }
            return false;
        }
        return $articleContent;
    }
    /**
     * Get the inner text of a node.
     * This also strips out any excess whitespace to be found.
@@ -937,7 +1030,9 @@ class Readability
        if (!isset($e->textContent) || $e->textContent === '') {
            return '';
        }
        $textContent = trim($e->textContent);
        if ($flattenLines) {
            $textContent = mb_ereg_replace('(?:[\r\n](?:\s|&nbsp;)*)+', '', $textContent);
        } elseif ($normalizeSpaces) {
@@ -946,6 +1041,7 @@ class Readability
        return $textContent;
    }
    /**
     * Remove the style attribute on every $e and under.
     *
@@ -956,11 +1052,14 @@ class Readability
        if (!is_object($e)) {
            return;
        }
        $elems = $e->getElementsByTagName('*');
        foreach ($elems as $elem) {
            $elem->removeAttribute('style');
        }
    }
    /**
     * Get comma number for a given text.
     *
@@ -972,6 +1071,7 @@ class Readability
    {
        return substr_count($text, ',');
    }
    /**
     * Get words number for a given text if words separated by a space.
     * Input string should be normalized.
@@ -984,6 +1084,7 @@ class Readability
    {
        return substr_count($text, ' ');
    }
    /**
     * Get the density of links as a percentage of the content
     * This is the amount of text that is inside a link divided by the total text in the node.
@@ -999,18 +1100,21 @@ class Readability
        $links = $e->getElementsByTagName('a');
        $textLength = mb_strlen($this->getInnerText($e, true, true));
        $linkLength = 0;
-        for ($dRe = $this->domainRegExp, $i = 0, $il = $links->length; $i < $il; $i++) {
+
        for ($dRe = $this->domainRegExp, $i = 0, $il = $links->length; $i < $il; ++$i) {
            if ($excludeExternal && $dRe && !preg_match($dRe, $links->item($i)->getAttribute('href'))) {
                continue;
            }
            $linkLength += mb_strlen($this->getInnerText($links->item($i)));
        }
        if ($textLength > 0 && $linkLength > 0) {
            return $linkLength / $textLength;
        } else {
            return 0;
        }
        return 0;
    }
    /**
     * Get an element weight by attribute.
     * Uses regular expressions to tell if this element looks good or bad.
@@ -1026,6 +1130,7 @@ class Readability
            return 0;
        }
        $weight = 0;
        //$attribute_val = trim($element->getAttribute('class')." ".$element->getAttribute('id'));
        $attribute_val = trim($element->getAttribute($attribute));
        if ($attribute_val != '') {
@@ -1045,6 +1150,7 @@ class Readability
        return $weight;
    }
    /**
     * Get an element relative weight.
     *
@@ -1057,6 +1163,7 @@ class Readability
        if (!$this->flagIsActive(self::FLAG_WEIGHT_ATTRIBUTES)) {
            return 0;
        }
        $weight = 0;
        /* Look for a special classname */
        $weight += $this->weightAttribute($e, 'class');
@@ -1065,6 +1172,7 @@ class Readability
        return $weight;
    }
    /**
     * Remove extraneous break tags from a node.
     *
@@ -1076,6 +1184,7 @@ class Readability
        $html = preg_replace($this->regexps['killBreaks'], '<br />', $html);
        $node->innerHTML = $html;
    }
    /**
     * Clean a node of all elements of type "tag".
     * (Unless it's a youtube/vimeo video. People love movies.).
@@ -1089,23 +1198,29 @@ class Readability
    {
        $targetList = $e->getElementsByTagName($tag);
        $isEmbed = ($tag === 'audio' || $tag === 'video' || $tag === 'iframe' || $tag === 'object' || $tag === 'embed');
-        for ($cur_item = null, $y = $targetList->length - 1; $y >= 0; $y--) {
+
        for ($cur_item = null, $y = $targetList->length - 1; $y >= 0; --$y) {
            /* Allow youtube and vimeo videos through as people usually want to see those. */
            $cur_item = $targetList->item($y);
            if ($isEmbed) {
                $attributeValues = $cur_item->getAttribute('src').' '.$cur_item->getAttribute('href');
                /* First, check the elements attributes to see if any of them contain known media hosts */
                if (preg_match($this->regexps['media'], $attributeValues)) {
                    continue;
                }
                /* Then check the elements inside this element for the same. */
                if (preg_match($this->regexps['media'], $targetList->item($y)->innerHTML)) {
                    continue;
                }
            }
            $cur_item->parentNode->removeChild($cur_item);
        }
    }
    /**
     * Clean an element of all tags of type "tag" if they look fishy.
     * "Fishy" is an algorithm based on content length, classnames,
@@ -1119,20 +1234,23 @@ class Readability
        if (!$this->flagIsActive(self::FLAG_CLEAN_CONDITIONALLY)) {
            return;
        }
        $tagsList = $e->getElementsByTagName($tag);
        $curTagsLength = $tagsList->length;
        /*
         * Gather counts for other typical elements embedded within.
         * Traverse backwards so we can remove nodes at the same time without effecting the traversal.
         *
         * TODO: Consider taking into account original contentScore here.
         */
-        for ($node = null, $i = $curTagsLength - 1; $i >= 0; $i--) {
+        for ($node = null, $i = $curTagsLength - 1; $i >= 0; --$i) {
            $node = $tagsList->item($i);
            //$class = $node->getAttribute('class').' '.$node->getAttribute('id'); //debug
            $weight = $this->getWeight($node);
            $contentScore = ($node->hasAttribute('readability')) ? (int) $node->getAttribute('readability') : 0;
            $this->dbg('Start conditional cleaning of '.$node->getNodePath().' (class='.$node->getAttribute('class').'; id='.$node->getAttribute('id').')'.(($node->hasAttribute('readability')) ? (' with score '.$node->getAttribute('readability')) : ''));
            if ($weight + $contentScore < 0) {
                $this->dbg('Removing...');
                $node->parentNode->removeChild($node);
@@ -1148,20 +1266,24 @@ class Readability
                $a = $node->getElementsByTagName('a')->length;
                $embedCount = 0;
                $embeds = $node->getElementsByTagName('embed');
-                for ($ei = 0, $il = $embeds->length; $ei < $il; $ei++) {
+
                for ($ei = 0, $il = $embeds->length; $ei < $il; ++$ei) {
                    if (preg_match($this->regexps['media'], $embeds->item($ei)->getAttribute('src'))) {
-                        $embedCount++;
+                        ++$embedCount;
                    }
                }
                $embeds = $node->getElementsByTagName('iframe');
-                for ($ei = 0, $il = $embeds->length; $ei < $il; $ei++) {
+                for ($ei = 0, $il = $embeds->length; $ei < $il; ++$ei) {
                    if (preg_match($this->regexps['media'], $embeds->item($ei)->getAttribute('src'))) {
-                        $embedCount++;
+                        ++$embedCount;
                    }
                }
-                $linkDensity   = $this->getLinkDensity($node, true);
+
                $linkDensity = $this->getLinkDensity($node, true);
                $contentLength = mb_strlen($this->getInnerText($node));
                $toRemove = false;
                if ($this->lightClean) {
                    if ($li > $p && $tag != 'ul' && $tag != 'ol') {
                        $this->dbg(' too many <li> elements, and parent is not <ul> or <ol>');
@@ -1192,8 +1314,8 @@ class Readability
                    } elseif ($input > floor($p / 3)) {
                        $this->dbg('  too many <input> elements');
                        $toRemove = true;
-                    } elseif ($contentLength < 25 && ($img === 0 || $img > 2)) {
+                    } elseif ($contentLength < 10 && ($img === 0 || $img > 2)) {
-                        $this->dbg('  content length less than 25 chars and 0 images, or more than 2 images');
+                        $this->dbg('  content length less than 10 chars and 0 images, or more than 2 images');
                        $toRemove = true;
                    } elseif ($weight < 25 && $linkDensity > 0.2) {
                        $this->dbg('  weight is '.$weight.' lower than 0 and link density is '.sprintf('%.2f', $linkDensity).' > 0.2');
@@ -1206,6 +1328,7 @@ class Readability
                        $toRemove = true;
                    }
                }
                if ($toRemove) {
                    //$this->dbg('Removing: '.$node->innerHTML);
                    $this->dbg('Removing...');
@@ -1214,6 +1337,7 @@ class Readability
            }
        }
    }
    /**
     * Clean out spurious headers from an Element. Checks things like classnames and link density.
     *
@@ -1221,25 +1345,39 @@ class Readability
     */
    public function cleanHeaders($e)
    {
-        for ($headerIndex = 1; $headerIndex < 3; $headerIndex++) {
+        for ($headerIndex = 1; $headerIndex < 3; ++$headerIndex) {
            $headers = $e->getElementsByTagName('h'.$headerIndex);
-            for ($i = $headers->length - 1; $i >= 0; $i--) {
+            for ($i = $headers->length - 1; $i >= 0; --$i) {
                if ($this->getWeight($headers->item($i)) < 0 || $this->getLinkDensity($headers->item($i)) > 0.33) {
                    $headers->item($i)->parentNode->removeChild($headers->item($i));
                }
            }
        }
    }
    public function flagIsActive($flag)
    {
        return ($this->flags & $flag) > 0;
    }
    public function addFlag($flag)
    {
        $this->flags = $this->flags | $flag;
    }
    public function removeFlag($flag)
    {
        $this->flags = $this->flags & ~$flag;
    }
    /**
     * Will recreate previously deleted body property.
     */
    protected function reinitBody()
    {
        if (!isset($this->body->childNodes)) {
            $this->body = $this->dom->createElement('body');
            $this->body->innerHTML = $this->bodyCache;
        }
    }
 }
@@ -19,6 +19,9 @@ class ReadabilityTested extends Readability
 class ReadabilityTest extends \PHPUnit_Framework_TestCase
 {
    /**
     * @requires extension tidy
     */
    public function testConstructDefault()
    {
        $readability = new ReadabilityTested('');
@@ -30,6 +33,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertInstanceOf('DomDocument', $readability->dom);
    }
    /**
     * @requires extension tidy
     */
    public function testConstructSimple()
    {
        $readability = new ReadabilityTested('<html/>', 'http://0.0.0.0');
@@ -41,6 +47,28 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertInstanceOf('DomDocument', $readability->dom);
    }
    public function testConstructDefaultWithoutTidy()
    {
        $readability = new ReadabilityTested('', null, 'libxml', false);
        $this->assertNull($readability->url);
        $this->assertContains('Parsing URL', $readability->getDebugText());
        $this->assertNotContains('Tidying document', $readability->getDebugText());
        $this->assertNull($readability->getDomainRegexp());
        $this->assertInstanceOf('DomDocument', $readability->dom);
    }
    public function testConstructSimpleWithoutTidy()
    {
        $readability = new ReadabilityTested('<html/>', 'http://0.0.0.0', 'libxml', false);
        $this->assertEquals('http://0.0.0.0', $readability->url);
        $this->assertContains('Parsing URL', $readability->getDebugText());
        $this->assertNotContains('Tidying document', $readability->getDebugText());
        $this->assertEquals('/0\.0\.0\.0/', $readability->getDomainRegexp());
        $this->assertInstanceOf('DomDocument', $readability->dom);
    }
    public function testInitNoContent()
    {
        $readability = new ReadabilityTested('<html/>', 'http://0.0.0.0');
@@ -48,6 +76,8 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertFalse($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('Sorry, Readability was unable to parse this page for content.', $readability->getContent()->innerHTML);
    }
@@ -58,7 +88,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('<div readability=', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is the awesome content :)', $readability->getContent()->innerHTML);
    }
@@ -69,7 +101,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('<div readability=', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is the awesome content :)', $readability->getContent()->innerHTML);
    }
@@ -81,7 +115,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('<div readability=', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is the awesome content :)', $readability->getContent()->innerHTML);
    }
@@ -94,7 +130,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('<div readability=', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertContains('readabilityFootnoteLink', $readability->getContent()->innerHTML);
        $this->assertContains('readabilityLink-3', $readability->getContent()->innerHTML);
@@ -102,16 +140,18 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
    public function testStandardClean()
    {
-        $readability = new ReadabilityTested('<div><h2>Title</h2>'.str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7).'<a href="#nofollow" rel="nofollow">will be removed</a></div>', 'http://0.0.0.0');
+        $readability = new ReadabilityTested('<div><h2>Title</h2>'.str_repeat('<p>This is an awesome text with some links, here there are: <a href="http://0.0.0.0/test.html">the awesome</a></p>', 7).'<a href="#nofollow" rel="nofollow">will NOT be removed</a></div>', 'http://0.0.0.0');
        $readability->debug = true;
        $readability->lightClean = false;
        $res = $readability->init();
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('<div readability=', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
-        $this->assertNotContains('will be removed', $readability->getContent()->innerHTML);
+        $this->assertContains('will NOT be removed', $readability->getContent()->innerHTML);
        $this->assertNotContains('<h2>', $readability->getContent()->innerHTML);
    }
@@ -123,7 +163,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('<div readability=', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertContains('nofollow', $readability->getContent()->innerHTML);
    }
@@ -136,7 +178,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="article"', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertContains('nofollow', $readability->getContent()->innerHTML);
    }
@@ -149,7 +193,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="article"', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertNotContains('<aside>', $readability->getContent()->innerHTML);
        $this->assertContains('<footer/>', $readability->getContent()->innerHTML);
@@ -163,7 +209,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="article"', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertNotContains('This text should be removed', $readability->getContent()->innerHTML);
    }
@@ -176,7 +224,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="tr"', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
    }
@@ -188,7 +238,9 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="article"', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
    }
@@ -201,7 +253,69 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="article"', $readability->getContent()->innerHTML);
        $this->assertEmpty($readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
    }
    public function testTitle()
    {
        $readability = new ReadabilityTested('<title>this is my title</title><article class="awesomecontent">'.str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7).'<p></p></article>', 'http://0.0.0.0');
        $readability->debug = true;
        $res = $readability->init();
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="article"', $readability->getContent()->innerHTML);
        $this->assertEquals('this is my title', $readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
    }
    public function testTitleWithDash()
    {
        $readability = new ReadabilityTested('<title> title2 - title3 </title><article class="awesomecontent">'.str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7).'<p></p></article>', 'http://0.0.0.0');
        $readability->debug = true;
        $res = $readability->init();
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="article"', $readability->getContent()->innerHTML);
        $this->assertEquals('title2 - title3', $readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
    }
    public function testTitleWithDoubleDot()
    {
        $readability = new ReadabilityTested('<title> title2 : title3 </title><article class="awesomecontent">'.str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7).'<p></p></article>', 'http://0.0.0.0');
        $readability->debug = true;
        $res = $readability->init();
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="article"', $readability->getContent()->innerHTML);
        $this->assertEquals('title2 : title3', $readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
    }
    public function testTitleTooShortUseH1()
    {
        $readability = new ReadabilityTested('<title>too short</title><h1>this is my h1 title !</h1><article class="awesomecontent">'.str_repeat('<p>This is an awesome text with some links, here there are the awesome</p>', 7).'<p></p></article>', 'http://0.0.0.0');
        $readability->debug = true;
        $res = $readability->init();
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('alt="article"', $readability->getContent()->innerHTML);
        $this->assertEquals('this is my h1 title !', $readability->getTitle()->innerHTML);
        $this->assertContains('This is an awesome text with some links, here there are', $readability->getContent()->innerHTML);
        $this->assertNotContains('This text is also an awesome text and you should know that', $readability->getContent()->innerHTML);
    }
@@ -216,4 +330,53 @@ class ReadabilityTest extends \PHPUnit_Framework_TestCase
    //     $this->assertEquals('/0\.0\.0\.0/', $readability->getDomainRegexp());
    //     $this->assertInstanceOf('DomDocument', $readability->dom);
    // }
    // dummy function to be used to the next test
    public function error2Exception($code, $string, $file, $line, $context)
    {
        throw new \Exception($string, $code);
    }
    public function testAutoClosingIframeNotThrowingException()
    {
        error_reporting(E_ALL | E_STRICT);
        ini_set('display_errors', true);
        set_error_handler(array($this, 'error2Exception'), E_ALL | E_STRICT);
        $data = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
            <html xmlns="http://www.w3.org/1999/xhtml" lang="ru-RU" prefix="og: http://ogp.me/ns#">
            <head profile="http://gmpg.org/xfn/11">
            <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
            <meta name="viewport" content="width=device-width, initial-scale=1.0" />
            </head>
            <body class="single single-post postid-22030 single-format-standard">
              <div id="wrapper">
                <div id="content">
                  <div class="post-22030 post type-post status-publish format-standard has-post-thumbnail hentry category-video category-reviews tag-193" id="post-22030">
                    <h1>3D Touch &#8212; будущее мобильных игр</h1>
                    <div class="postdate">Автор:  <strong>Сергей Пак</strong> | Просмотров: 1363 | Опубликовано:  14 сентября 2015 </div>
                    <div class="entry">
                      <p>Компания Apple представила новую технологию 3D Touch, которая является прямым потомком более ранней версии Force Touch &#8212; последняя, напомним, используется сейчас в трекпадах Macbook Pro и Macbook 2015. Теперь управлять устройством стало в разы проще, и Force Touch открывает перед пользователями новые возможности, но при этом 3D Touch &#8212; это про другое. Дело в том, что теперь и на мобильных устройствах интерфейс будет постепенно меняться, кардинальные перемены ждут мобильный гейминг, потому что здесь разработчики действительно могут разгуляться.<span id="more-22030"></span></p>
                      <p><iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p>
                      <p>Итак, просто представьте себе, что iPhone 6S &#8212; это, по большому счету, отличная игровая приставка, которую вы носите с собой, а еще она может выдавать невероятной красоты картинку. Но проблема заключается, пожалуй, в том, что управлять персонажем в играх довольно трудно &#8212; он неповоротлив, обладает заторможенной реакцией, а игровой клиент зачастую требует перегруза интерфейса для того, чтобы обеспечить максимально большое количество возможностей. Благодаря трехуровневому нажатию можно избавиться от лишних кнопок и обеспечить более качественный обзор местности, и при этом пользователь будет закрывать пальцами минимальное пространство.</p>
                    </div>
                  </div>
                </div>
              </div>
            </body>
            </html>';
        $readability = new ReadabilityTested($data, 'http://iosgames.ru/?p=22030');
        $readability->debug = true;
        $res = $readability->init();
        $this->assertTrue($res);
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getContent());
        $this->assertInstanceOf('Readability\JSLikeHTMLElement', $readability->getTitle());
        $this->assertContains('<iframe src="https://www.youtube.com/embed/PUep6xNeKjA" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"> </iframe>', $readability->getContent()->innerHTML);
        $this->assertContains('3D Touch', $readability->getTitle()->innerHTML);
    }
 }
Author	SHA1	Message	Date
Jeremy Benoist	41d7440c6e	Merge pull request #10 from j0k3r/remove-tidy-constraint Remove tidy from requirement	2015-11-10 09:55:29 +01:00
Jeremy Benoist	252bb4ef42	Add Tidy requirement in README	2015-11-09 20:25:39 +01:00
Jeremy Benoist	7c30d76b6e	Ensure tests are running without Tidy	2015-11-09 19:59:18 +01:00
Jeremy Benoist	111cb08034	Improve negative element - add unlikelyCandidates: head - add negative: recommend	2015-11-09 19:44:11 +01:00
Jeremy Benoist	74fbf6f009	Remove tidy from requirement Sadly we can't suggest user to install the extension	2015-11-09 11:12:07 +01:00
Jeremy Benoist	1565a6819a	Add .gitattributes to slim down composer packages When composer is used as a dependency manager for large web apps, transferring lots of extra files (such as tests and documentation) can be very time and bandwidth consuming. By using .gitattributes we can tell composer what files to ignore when distributing through composer.	2015-10-29 13:56:13 +01:00
Jeremy Benoist	f71c3a4196	Do not remove html tag attributes They might contains useful information (at least language)	2015-09-23 21:09:38 +02:00
Jeremy Benoist	255a2fc7bc	Merge pull request #8 from j0k3r/nofollow Do not remove nofollow links	2015-09-23 17:58:32 +02:00
Jeremy Benoist	b77876b30a	Do not remove nofollow links Most the time, they can be usefull. At least, it'll be a link to something unrelated. But we won't lose a link inside the content. Also, adding some extra space.	2015-09-22 19:25:57 +02:00
Jeremy Benoist	1830dc45d4	Merge pull request #7 from j0k3r/fix-nbsp Avoid error with	2015-09-20 21:05:55 +02:00
Jeremy Benoist	6be1f9b984	Fix link to fivefilters fork	2015-09-18 19:19:26 +02:00
Jeremy Benoist	175196d6c2	Avoid error with   Fix #5	2015-09-18 19:10:48 +02:00
Jeremy Benoist	2b5af601d5	Do not format output to avoid breaking apps It'll require to jump to 2.0.0 and I think it's to soon	2015-09-15 22:25:08 +02:00
Jeremy Benoist	d01eb2ac1e	Use class instead of id to avoid error It generates error like `ID XXX already defined`	2015-09-14 21:49:40 +02:00
Jeremy Benoist	c5a4a490e1	CS	2015-08-24 11:10:54 +02:00
Jeremy Benoist	908a49824f	Add test on title	2015-08-24 11:09:47 +02:00
Jeremy Benoist	c67189248e	Backport changes from wallabag https://github.com/wallabag/php-readability/commit/e9e4ff87f8fc56d406ccdd5a9a7f1d3d6af07e79	2015-08-24 11:09:38 +02:00