HaishinKit.kt

Introduction: [WIP]Camera and Microphone streaming library via RTMP for Android.
More: Author   ReportBugs   
Tags:

GitHub license GitHub Sponsor

💖 Sponsors

Do you need additional support? Technical support on Issues and Discussions is provided only to contributors and academic researchers of HaishinKit. By becoming a sponsor, we can provide the support you need.

Sponsor: $50 per month: Technical support via GitHub Issues/Discussions with priority response.

💬 Communication

  • GitHub Issues and Discussions are open spaces for communication among users and are available to everyone as long as the code of conduct is followed.
  • Whether someone is a contributor to HaishinKit is mainly determined by their GitHub profile icon. If you are using the default icon, there is a chance your input might be overlooked, so please consider setting a custom one. It could be a picture of your pet, for example. Personally, I like cats.
  • If you want to support e-mail based communication without GitHub.
    • Consulting fee is $50/1 incident. I'm able to response a few days.
Project name Notes License
HaishinKit for iOS, macOS, tvOS and visionOS. Camera and Microphone streaming library via RTMP for Android. BSD 3-Clause "New" or "Revised" License
HaishinKit for Flutter. Camera and Microphone streaming library via RTMP for Flutter. BSD 3-Clause "New" or "Revised" License

🎨 Features

RTMP

  • [x] Authentication
  • [x] Publish
  • [x] Playback
  • [ ] Action Message Format
    • [x] AMF0
    • [ ] AMF3
  • [ ] SharedObject
  • [x] RTMPS
    • [x] Native (RTMP over SSL/TSL)
  • [ ] Enhanced RTMP (Working in progress)
    • [ ] v1
    • [ ] v2
  • [x] Audio Codecs
    • [x] AAC
  • [x] Video Codecs
    • [x] H264, HEVC

⏺️ Recording

Now support local recording. Additionally, you can specify separate videoSettings and audioSettings from the live stream.

var mixer = MediaMixer(context)
var recorder = MediaRecorder(context)
mixer.registerOutput(recorder)
recorder.videoSettings.profileLevel = VideoCodecProfileLevel.HEVC_MAIN_3_1
recorder.startRecording(
    File(
        Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS),
        "output.mp4"
    ).toString(),
    MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4
)

Filter

Sources

  • [x] Single camera with Camera2 api
  • [x] Multi camera with Camera2 api
  • [x] MediaProjection
  • [x] Microphone with AudioRecord api.

View rendering

- HkSurfaceView HkTextureView
Engine SurfaceView TextureView
Playback beta beta
Publish ✅ Stable ✅ Stable
Note Recommend Android 7.0+ Recommend Android 5.0-6.0

Others

  • [x] Hardware acceleration for H264 video encoding/AAC audio encoding.
    • [x] Asynchronously processing.
  • [x] Graphics api
    • [x] ✅ OpenGL
    • [ ] 🐛 Vulkan

Settings

stream.audioSettings.bitrate = 32 * 1000

stream.videoSettings.width = 640 // The width resolution of video output.
stream.videoSettings.height = 360 // The height resolution of video output.
stream.videoSettings.bitrate = 160 * 1000 // The bitRate of video output.
stream.videoSettings.IFrameInterval = 2 // The keyframe interval

Offscreen Rendering.

Through off-screen rendering capabilities, it is possible to display any text or bitmap on a video during broadcasting or viewing. This allows for various applications such as watermarking and time display.

mixer.attachVideo(0, cameraSource)

val text = TextScreenObject()
text.textSize = 60f
text.textValue = "23:44:56"
text.layoutMargins.set(0, 0, 16, 16)
text.horizontalAlignment = ScreenObject.HORIZONTAL_ALIGNMENT_RIGHT
text.verticalAlignment = ScreenObject.VERTICAL_ALIGNMENT_BOTTOM
mixer.screen.addChild(text)

val image = ImageScreenObject()
image.bitmap = BitmapFactory.decodeResource(resources, R.drawable.game_jikkyou)
image.verticalAlignment = ScreenObject.VERTICAL_ALIGNMENT_BOTTOM
image.frame.set(0, 0, 180, 180)
mixer.screen.addChild(image)

🐾 Examples

Examples project are available for Android.

  • [x] Camera and microphone publish.
  • [x] RTMP Playback
git clone https://github.com/HaishinKit/HaishinKit.kt.git
cd HaishinKit.kt

# Open [Android Studio] -> [Open] ...

🔧 Usage

Gradle dependency

JitPack

  • A common mistake is trying to use implementation 'com.github.HaishinKit.HaishinKit.kt', which does not work. The correct form is implementation 'com.github.HaishinKit.HaishinKit~kt'.
  • In older versions, there may be cases where Jetpack is not supported. If it's not available, please give up and use the latest version.
allprojects {
  repositories {
    maven { url 'https://jitpack.io' }
  }
}

dependencies {
  implementation 'com.github.HaishinKit.HaishinKit~kt:haishinkit:x.x.x'
  implementation 'com.github.HaishinKit.HaishinKit~kt:rtmp:x.x.x'
  implementation 'com.github.HaishinKit.HaishinKit~kt:compose:x.x.x'
  implementation 'com.github.HaishinKit.HaishinKit~kt:lottie:x.x.x'
}

Dependencies

- minSdk Android Requirements Status Description
haishinkit 21+ 5 Require Stable It's the base module for HaishinKit.
rtmp 21+ 5 Require Stable It's support for an rtmp streaming.
compose 21+ 5 Optional Beta It's support for a composable component for HaishinKit.
lottie 21+ 5 Optional Beta It's a module for embedding Lottie animations into live streaming video.

Android manifest


<uses-permission android:name="android.permission.INTERNET" /><uses-permission
android:name="android.permission.CAMERA" /><uses-permission
android:name="android.permission.RECORD_AUDIO" />

Prerequisites

ActivityCompat.requestPermissions(
    this, arrayOf(
        Manifest.permission.CAMERA,
        Manifest.permission.RECORD_AUDIO
    ), 1
)

RTMP Usage

Real Time Messaging Protocol (RTMP).

Filter API (v0.1)

- [assets]
  - [shaders]
    - custom-shader.vert(optional)
    - custom-shader.frag
package my.custom.filter

import com.haishinkit.graphics.filter.VideoEffect

class Monochrome2VideoEffect(
    override val name: String = "custom-shader"
) : VideoEffect
mixer.screen.videoEffect = Monochrome2VideoEffect()

📜 License

BSD-3-Clause

Apps
About Me
GitHub: Trinea
Facebook: Dev Tools