Flux Controlnet

flux-dev-controlnet

Flux Controlnet provides precise adjustments for image generation tasks, enhancing creativity and control.

Partner Model

A100 80GB

Fast Inference

REST API

Try in Console API Docs Examples

Model Information

Response Time~39 sec

StatusActive

Version

0.0.1

Updatedabout 1 hour ago

Live Demo

Average runtime: ~39 seconds

Input

Configure model parameters

Negative Prompt

Things you do not want to see in your image

low quality, ugly, distorted, artefacts

Prompt

a cyberpunk with natural greys and whites and browns

control_type

An enumeration.

Control Strength

Strength of control net. Different controls work better with different strengths. Canny works best with 0.5, soft edge works best with 0.4, and depth works best between 0.5 and 0.75. If images are low quality, try reducing the strength and try reducing the guidance scale.

0.45

Control Image

Image to use with control net

File upload is currently disabled

JPEGPNGJPGWEBP

Lora Url

Optional LoRA model to use. Give a URL to a HuggingFace .safetensors file, a Replicate .tar file or a CivitAI download link.

Enter your lora url

Lora Strength

Strength of LoRA model

Enter your lora strength

output_format

An enumeration.

Output

View generated results

Result

Preview, share or download your results with a single click.

Cost is calculated based on execution time.The model is charged at $0.002 per second. With a $1 budget, you can run this model approximately 12 times, assuming an average execution time of 39 seconds per run.

API Reference

View Full Documentation

Prerequisites

Create an API Key from the Eachlabs Console
Install the required dependencies for your chosen language (e.g., requests for Python)

API Integration Steps

1. Create a Prediction

Send a POST request to create a new prediction. This will return a prediction ID that you'll use to check the result. The request should include your model inputs and API key.

import requests
import time

API_KEY = "YOUR_API_KEY"  # Replace with your API key
HEADERS = {
    "X-API-Key": API_KEY,
    "Content-Type": "application/json"
}

def create_prediction():
    response = requests.post(
        "https://api.eachlabs.ai/v1/prediction/",
        headers=HEADERS,
        json={
            "model": "flux-dev-controlnet",
            "version": "0.0.1",
            "input": {
  "seed": null,
  "steps": "28",
  "prompt": "your prompt here",
  "lora_url": "your lora url here",
  "control_type": "depth",
  "control_image": "your_file.image/jpeg",
  "lora_strength": "1",
  "output_format": "webp",
  "guidance_scale": "3.5",
  "output_quality": "80",
  "negative_prompt": "your negative prompt here",
  "control_strength": "0.5",
  "depth_preprocessor": "DepthAnything",
  "soft_edge_preprocessor": "HED",
  "image_to_image_strength": "0",
  "return_preprocessed_image": false
}
        }
    )
    prediction = response.json()
    
    if prediction["status"] != "success":
        raise Exception(f"Prediction failed: {prediction}")
    
    return prediction["predictionID"]

2. Get Prediction Result

Poll the prediction endpoint with the prediction ID until the result is ready. The API uses long-polling, so you'll need to repeatedly check until you receive a success status.

def get_prediction(prediction_id):
    while True:
        result = requests.get(
            f"https://api.eachlabs.ai/v1/prediction/{prediction_id}",
            headers=HEADERS
        ).json()
        
        if result["status"] == "success":
            return result
        elif result["status"] == "error":
            raise Exception(f"Prediction failed: {result}")
        
        time.sleep(1)  # Wait before polling again

3. Complete Example

Here's a complete example that puts it all together, including error handling and result processing. This shows how to create a prediction and wait for the result in a production environment.

try:
    # Create prediction
    prediction_id = create_prediction()
    print(f"Prediction created: {prediction_id}")
    
    # Get result
    result = get_prediction(prediction_id)
    print(f"Output URL: {result['output']}")
    print(f"Processing time: {result['metrics']['predict_time']}s")
except Exception as e:
    print(f"Error: {e}")

Additional Information

The API uses a two-step process: create prediction and poll for results
Response time: ~39 seconds
Rate limit: 60 requests/minute
Concurrent requests: 10 maximum
Use long-polling to check prediction status until completion

Overview

Flux Controlnet is a cutting-edge model designed for generating high-quality images with precision and customization. By utilizing various preprocessor settings, control types, and parameter adjustments, users can achieve detailed and creative outputs tailored to their specific needs. This document provides essential details to help users effectively interact with Flux Controlnet, understand its capabilities, and optimize their usage.

Technical Specifications

Control Types:

Canny: Detects edges in images, focusing on clear outlines.
Soft Edge: Captures smoother edges for a softer and more artistic style.
Depth: Utilizes depth-based processing for a more realistic 3D-like rendering.

Preprocessors:

Depth Preprocessors: Midas, Zoe, DepthAnything, Zoe-DepthAnything.
Soft Edge Preprocessors: HED, TEED, PiDiNet.

Customizable Parameters for Flux Controlnet:

Guidance Scale: Defines the adherence to the input prompt .
Steps: Determines the number of iterations for image generation .
Control Strength: Adjusts the effect strength of the control type.
Image-to-Image Strength: Balances between original input image and generated transformations .

Key Considerations

Prompt Quality for Flux Controlnet: Ensure the prompt is descriptive and relevant to your desired output. Avoid vague descriptions for better results.

Control Image: When using control_image, provide high-quality images that match the control type (e.g., clear edges for canny).

Preprocessor Compatibility: Select preprocessors that align with your control type. For example, use HED or PiDiNet with Soft Edge.

Lora Parameters: Use lora_strength and lora_url to incorporate specific weights or styles for further customization.

Tips & Tricks

Guidance Scale:
- Use lower values (e.g., 1-2) for more freedom in artistic creativity.
- Higher values (e.g., 4-5) ensure stronger adherence to the prompt but might limit flexibility.
Steps:
- For quick drafts or initial ideas, set steps between 5-15.
- For detailed results, use higher values like 30-50, keeping in mind that processing time increases with higher steps.
Control Strength:
- Set to 1 for a balanced effect.
- Use values closer to 3 for more pronounced control but avoid overuse as it may distort the image.
Image-to-Image Strength:
- Keep values near 0.5 for a balanced blend between the input and generated image.
- Lower values (e.g., 0.2) prioritize the generated content, while higher values (e.g., 0.8) retain more of the original image.
Depth Preprocessors:
- Use Midas for general depth mapping.
- Select Zoe or DepthAnything for scenes with complex layers.
Soft Edge Preprocessors:
- Choose HED for clean and defined edges.
- Use TEED or PiDiNet for a softer, stylized edge effect.
Lora Strength:
- Start with a value of 2 for balanced adjustments.
- Increase to 3 for stronger stylistic emphasis.
Output Quality:
- For web usage, values between 70-85 are sufficient.
- For detailed prints or high-resolution purposes, set values closer to 100.

Capabilities

Flux Controlnet generates photorealistic images with enhanced depth and edge controls.

Stylized outputs by leveraging lora_url for external weight influences.

Balancing creativity and precision through a wide range of customizable inputs.

What can I use for?

Creating customized artwork with precise depth and edge control.

Enhancing existing images by applying transformations and refinements.

Developing assets for creative projects, including visual storytelling and design with Flux Controlnet.

Things to be aware of

Combine soft_edge with HED for a clean, comic-style effect.

Experiment with DepthAnything in complex landscapes to highlight depth details.

Use a high image_to_image_strength value (e.g., 0.9) for minor touch-ups on existing images.

Limitations

Excessively high steps or guidance_scale values may result in processing delays or unnatural outputs.

Compatibility between control_type and preprocessors must be carefully managed to avoid suboptimal results.

Lower quality control images may lead to poor image outputs, even with optimized parameters.

Output Format: WEBP,JPG,PNG

Related AI Models

Flux Depth Pro

flux-depth-pro

Image to Image

Flux Fill Pro

flux-fill-pro

Image to Image

Eachlabs Face Swap

each-faceswap-v1

Image to Image

Style Changer by ByteDance

bytedance

Image to Image