Grading Rubric (Group Assignment)

Assessment of Learning Goal 1: Quality of describing how image processing can be used in the design process and Quality of the critical reflection of model capability (weight 45%)

• Excellent (9-10)
  • Take a set of photos that have very diverse conditions (e.g., weather, lighting, location, angle, scale, etc.)
  • Summarize the model output into convincing findings that can inform the design of the product-service system
  • Use rich examples with different variety to reflect on model capability in a great detail
  • Have convincing insights about how to create a good dataset for training models
• Good (7-8)
  • Take a set of photos that have reasonably diverse conditions (e.g., weather, lighting, location, angle, scale, etc.)
  • Summarize the model output into findings that can provide some useful insights in the design process
  • Use proper examples to reflect on model capability in a reasonable way
  • Have proper insights about how to create a good dataset for training models
• Sufficient (6)
  • Take a set of photos that may not have diverse conditions (e.g., weather, lighting, location, angle, scale, etc.)
  • Have some summarization of the findings from the model output, but some parts may not be convincing or not elaborated properly
  • Use some examples to reflect on model capability, but the reflection may have flaws
  • Have some insights about how to create datasets for training models, but some insights may not be convincing
• Insufficient (<6)
  • No photos or only a few photos are taken with poor diversity
  • Have no summarization of the findings from the model output, or the summarization has poor quality
  • No examples or no reflections on model capability, or the reflections have poor quality
  • Have no insights about dataset creation, or the insights have poor quality

Assessment of Learning Goal 2: Quality of problem identification, model training, and application design (weight 45%)

• Excellent (9-10)
  • Describe the problem with clear and precise details
  • Provide a thorough training procedure and explain how the data is utilized in the model
  • Assess the model performance in a detailed manner, including the number of correct and incorrect identifications
  • Design the application with a rich user interface, a detailed dataflow, and a clear flowchart
• Good (7-8)
  • Describe the problem with reasonable details
  • Provide a reasonable training procedure and explain how the data is utilized in the model
  • Assess the model performance with some details, including the number of correct and incorrect identifications
  • Design the application with a reasonable user interface, a reasonable dataflow, and a reasonable flowchart
• Sufficient (6)
  • Describe the problem with some details, but some parts may not be convincing
  • Provide some training procedures and explain how the data is utilized in the model, but some parts may not be convincing
  • Assess the model performance with some details, but some parts may not be convincing
  • Design the application with some user interface, a dataflow, and a flowchart, but some parts may not be convincing
• Insufficient (<6)
  • No description of the problem, or the description has poor quality
  • No training procedure or the training procedure has poor quality
  • No assessment of the model performance, or the assessment has poor quality
  • No design of the application, or the design has poor quality

Assessment of Learning Goal 3: Ability to automate the image processing pipeline (weight 10%)

• Excellent (9-10)
  • Fully automate the image processing pipeline
  • Have good documentation about how the code works
  • Have very good code quality
• Good (7-8)
  • Fully automate the image processing pipeline
  • Have reasonable documentation about how the code works
  • Most of the code is human-readable
• Sufficient (6)
  • Some image processing pipeline is automated but some parts require human effort
  • Have some documentation about how the code works, but some may not be clear
  • Some part of the code is hard to understand
• Insufficient (<6)
  • The image processing pipeline is not automated and requires all human effort to drag and drop the images into the web interface to get the results