luigi12345 commited on
Commit
a3da2c1
·
verified ·
1 Parent(s): bfdb85f

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +16 -60
app.py CHANGED
@@ -173,7 +173,7 @@ def generate_image(prompt,
173
 
174
  # Gradio interface
175
  with gr.Blocks() as demo:
176
- gr.Markdown(value="# Multimodal Understanding")
177
  with gr.Row():
178
  image_input = gr.Image()
179
  with gr.Column():
@@ -186,66 +186,22 @@ with gr.Blocks() as demo:
186
  understanding_output = gr.Textbox(label="Response")
187
 
188
  examples_inpainting = gr.Examples(
189
- label="Multimodal Understanding examples",
190
  examples=[
191
  [
192
- """Analyze the provided fundus image in exhaustive detail, following the standard ophthalmological protocol for fundus examination. Output an HTML report structured as a formal medical document. The report MUST:
193
-
194
- 1. **Image Quality Assessment:** Begin with a concise assessment of image quality, noting focus, illumination, field of view, and any artifacts (and their impact on assessability).
195
-
196
- 2. **Detailed Clinical Findings:** Describe each of the following areas with the utmost precision and specificity, using proper ophthalmological terminology:
197
- * **Optic Disc:**
198
- * Size and shape (including any abnormalities).
199
- * Color (specifically noting any pallor and its location).
200
- * Cup-to-Disc Ratio (CDR), providing both vertical and horizontal estimates.
201
- * Neuroretinal Rim: Assess rim thickness in all quadrants (superior, inferior, nasal, temporal). Explicitly state whether the ISNT rule is followed or violated. Describe any notching or focal thinning.
202
- * Peripapillary Region: Describe the presence/absence of peripapillary atrophy (PPA), differentiating between alpha and beta zones. Note any hemorrhages.
203
- * **Retinal Vasculature:**
204
- * Arterioles: Describe caliber (narrowing, dilation), tortuosity, and any focal abnormalities.
205
- * Venules: Describe caliber, tortuosity, and any abnormalities.
206
- * Arteriovenous (A/V) Ratio: Estimate the A/V ratio.
207
- * Crossing Changes: Note any arteriovenous nicking or other crossing abnormalities.
208
- * Vessel Course: Describe the course of the major vessels, and check for abnormalities.
209
- * **Macula:**
210
- * Foveal Reflex: Describe the presence/absence and quality of the foveal reflex.
211
- * Pigment Changes: Note any pigmentary abnormalities, drusen, or other lesions.
212
- * Edema/Exudates: Describe any signs of macular edema or exudates.
213
- * **Peripheral Retina:**
214
- * Mid-Periphery: Describe any abnormalities (hemorrhages, exudates, tears, etc.).
215
- * Far Periphery: Note the extent of visualization and any findings.
216
-
217
- 3. **Differential Diagnosis:** Based solely on the image findings, provide a prioritized differential diagnosis. Include the most likely diagnosis and any other plausible possibilities. For each diagnosis, explain the reasoning based on the observed features.
218
-
219
- 4. **Diagnostic Confidence:** Indicate the confidence level for the primary diagnosis. List the key image findings that support the diagnosis.
220
-
221
- 5. **Simulated AI Attention Metrics:** Create a table representing a *simulated* AI attention distribution. This should reflect the expected focus areas for the most likely diagnosis, based on the known importance of different features. Provide percentages for:
222
- * Optic Disc (Total)
223
- * Cup
224
- * Neuroretinal Rim (subdivided by region if significant differences exist)
225
- * Peripapillary Atrophy
226
- * Vessels
227
- * Macula
228
- * Periphery
229
-
230
- 6. **Summary and Impression:** Provide a concise summary of the key findings and the overall impression.
231
-
232
- 7. **Recommendations:**
233
- * Provide specific, actionable recommendations based on the image findings.
234
- * If referral is warranted, clearly state the urgency and the type of specialist.
235
- * List any recommended investigations (e.g., OCT, visual fields).
236
-
237
- 8. **Disclaimer:** Include a disclaimer stating that the report is based on image analysis alone and does not replace a full clinical examination.
238
-
239
- 9. **HTML Structure:** Use semantic HTML elements (h1-h3, p, ol, ul, table, div) to create a well-structured, readable report. Include:
240
- * A report header with a title ("EyeUnit.ai | AI for Ophthalmology") and a logo
241
- * Clearly labeled sections for each part of the analysis.
242
- * Tables for the "Overall Analysis Coverage" and "AI-Driven Attention Metrics."
243
-
244
- 10. **CSS Styling:** Apply CSS styles to make the report visually appealing and professional. The report should be suitable for both screen viewing and printing (use a `@media print` block to optimize for print).
245
- * **Crucial Details:**
246
- * **PATIENT ID, NAME, AGE and DATE OF EXAM**
247
-
248
- 11. **Crucial Details:** Output ONLY the complete HTML code. Do not provide any surrounding text or explanations. Focus solely on generating the HTML report.
249
  """ , "fundus.png",
250
  ],
251
  ],
@@ -253,7 +209,7 @@ with gr.Blocks() as demo:
253
  )
254
 
255
 
256
- gr.Markdown(value="# Text-to-Image Generation")
257
 
258
 
259
 
 
173
 
174
  # Gradio interface
175
  with gr.Blocks() as demo:
176
+ gr.Markdown(value="# AutoMultimodal")
177
  with gr.Row():
178
  image_input = gr.Image()
179
  with gr.Column():
 
186
  understanding_output = gr.Textbox(label="Response")
187
 
188
  examples_inpainting = gr.Examples(
189
+ label="Multimodal Medical Prompts",
190
  examples=[
191
  [
192
+ """
193
+ You are an AI assistant trained to analyze medical images. Analyze the attached fundus photograph in extreme detail, following a structured approach analogous to an ophthalmologist's examination. Provide a differential diagnosis solely based on this image analysis, without assuming any clinical history. Structure your response as follows:
194
+
195
+ Analysis Methodology (Concise): List, very briefly, the key anatomical areas/features you will assess, in the order of assessment (e.g., Optic Disc, Vessels, Macula, Periphery, Overall Quality). Do not describe the analysis process in detail here – just name the areas.
196
+ Detailed Image Analysis & Percentage Breakdown: Analyze the image, addressing each area listed in Step 1. For each area:
197
+ Provide a highly detailed, objective description, using precise ophthalmological terminology. Quantify observations whenever possible (e.g., cup-to-disc ratio, A/V ratio).
198
+ State the percentage of that area you were able to confidently analyze, based on image quality and clarity. For example: "Optic Disc: 90% analyzable (10% obscured by slight blurring at the superior margin)." "Macula: 100% analyzable." "Peripheral Retina (Nasal): 60% analyzable (40% not visible in the image)." Be precise.
199
+ For any areas where analysis is incomplete (<100%), briefly explain the limiting factor (e.g., poor focus, limited field of view, artifact).
200
+ Differential Diagnosis (Image-Based Only): Based solely on your Step 2 analysis, provide:
201
+ Most Likely Diagnosis (from image findings).
202
+ Other Possible Diagnoses (from image findings).
203
+ Rationale: For each diagnosis, briefly link specific image findings to the diagnostic criteria.
204
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
205
  """ , "fundus.png",
206
  ],
207
  ],
 
209
  )
210
 
211
 
212
+ gr.Markdown(value="# Text-to-Image Multimodal Generation")
213
 
214
 
215