RxRetroJsoup

Introduction: A simple API-like from html website (scrapper) for Android
More: Author   ReportBugs   
Tags:

RxJava2 ready !

Android app on Google Play

Create an interface with @Select annotated method

public interface TutosAndroidFrance {

    @Select("article")
    Observable<Article> articles();

}

Annotate your model with Jsoup queries ( https://jsoup.org/cookbook/extracting-data/selector-syntax )


public class Article {

    @JsoupText(".entry-title a")
    String title;

    @JsoupHref(".read-more a")
    String href;

    @JsoupSrc(".entry-thumb img")
    String image;

    @JsoupText(".entry-content p")
    String description;

    public Article() {
    }
    ...

}

Build a RetroJsoup

final TutosAndroidFrance tutosAndroidFrance = new RetroJsoup.Builder()
                .url("http://tutos-android-france.com/")

                //optionally
                .client(your_okhttp_client)

                .build()
                .create(TutosAndroidFrance.class);

Fetch your objects !

tutosAndroidFrance.articles()
                .toList()

                .subscribeOn(Schedulers.newThread())
                .observeOn(AndroidSchedulers.mainThread())

                .subscribe(items -> );

RxJsoup

RxJsoup.with(url)
    .select("article")
    .flatMap(element -> rxJsoup.text(element, ".entry-title a"))
    .subscribe(text -> {});


    .flatMap(element -> rxJsoup.src(element, ".entry img"))
    .flatMap(element -> rxJsoup.href(element, ".entry a"))
    .flatMap(element -> rxJsoup.attr(element, ".entry h1", "id"))
RxJsoup.connect(

           Jsoup.connect("www.thewebsite.com")
               .userAgent(MY_USER_AGENT)
               .data("credential", email)
               .data("pwd", password)
               .cookies(loginForm.cookies())
               .method(Connection.Method.POST)

           )
           .subscibe(response -> {})

Download

Buy Me a Coffee at ko-fi.com

In your module Download

compile 'com.github.florent37:retrojsoup:1.0.4'
compile 'com.github.florent37:rxjsoup:1.0.4'
annotationProcessor 'com.github.florent37:retrojsoup-compiler:1.0.4'

//don't forget to include jsoup & rxjava
compile 'org.jsoup:jsoup:1.10.2'
compile 'io.reactivex.rxjava2:rxjava:2.0.7'

//optionaly
compile 'com.squareup.okhttp3:okhttp:3.6.0'

Credits

Author: Florent Champigny http://www.florentchampigny.com/

Blog : http://www.tutos-android-france.com/

Android app on Google Play

Follow me on Google+ Follow me on Twitter Follow me on LinkedIn

License

Copyright 2017 florent37, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Support Me
Apps
About Me
Google+: Trinea trinea
GitHub: Trinea