| 0 | Questionperception | What are the important objects in the current scene? Those objects will be considered for the future reasoning and driving decision. |
| GT Answer | There is a sedan car to the front of the ego vehicle, a black SUV to the front of the ego vehicle, a slow down to the front of the ego vehicle, and a slow down to the front of the ego vehicle. The IDs of these objects are <c1,CAM_FRONT,937.5,495.8>, <c2,CAM_FRONT,864.5,525.0>, <c3,CAM_FRONT,1001.2,556.5>, and <c4,CAM_FRONT,705.8,423.5>. |
| InternVL2-2B | In the current scene, the important objects include:
1. The car in the foreground: The car is the primary object of interest. It is the closest object to the camera and is moving towards the right side of the frame. The car's position and movement are crucial for future reasoning and driving decisions.
2. The road: The road is the primary means of transportation in the scene. It is the path the car is following and is essential for navigation and decision-making.
3. The traffic lights: The traffic lights are visible on the left side of the road. They are important for ensuring safe driving and adhering to traffic rules.
4. The street signs: The street signs are visible on the right side of the road. They provide important information about the road's layout and direction.
5. The trees and foliage: The trees and foliage are visible on both sides of the road. They provide context and help in understanding the environment.
6. The streetlights: The streetlights are visible along the roadside. They are essential for visibility and safety during nighttime driving.
These objects are crucial for understanding the current scene and making informed driving decisions. |
| Mini-InternVL2-2B-DA-DriveLM | There is a black car to the front of the ego vehicle, a black sedan to the front of the ego vehicle, and a white road sign to the front of the ego vehicle. The IDs of these objects are 1, 2, and 3. |
| llama_adapter_v2_multimodal7b | In the current scene, the important objects to consider for future reasoning and driving decisions include the car, the traffic light, and the streetlight. The car is driving down the street, and the traffic light is present at the intersection, indicating when it is safe to proceed or stop. The streetlight provides illumination for the road, which is essential for visibility and safety. The presence of these objects will help the driver make informed decisions about their driving behavior, such as when to stop, when to proceed, and when to yield to other vehicles or pedestrians. |
| InternVL2-2b-LoRA-25k-drivelm | There is a black car to the front of the ego vehicle, a black car to the back of the ego vehicle, a black car to the front of the ego vehicle, and a red light to the front of the ego vehicle. The IDs of these objects are <c1,CAM_FRONT,1080.0,525.8>, <c2,CAM_BACK,1080.0,525.8>, <c3,CAM_FRONT,1080.0,505.8>, and <c4,CAM_FRONT,1080.0,465.8>. |
| InternVL2-2b-LoRA-25k-drivelm-offline-redcircle-ctag-bkgd | There is a black car to the front of the ego vehicle, a black car to the front of the ego vehicle, and a black car to the front of the ego vehicle. The IDs of these objects are <c1,CAM_FRONT,894.2,486.7>, <c2,CAM_FRONT,843.3,506.7>, and <c3,CAM_FRONT,1001.2,556.5>. |
| InternVL2-2b-LoRA-25k_plus_DL-PL-10pct | There is a car to the front of the ego vehicle, a car to the front of the ego vehicle, and a car to the front of the ego vehicle. The IDs of these objects are <c1,CAM_FRONT,1080.0,525.0>, <c2,CAM_FRONT,1080.0,525.0>, and <c3,CAM_FRONT,1080.0,525.0>. |
| InternVL2-2b-LoRA-300k-drivelm | There is a black car to the front of the ego vehicle, a black car to the back of the ego vehicle, a black car to the front of the ego vehicle, and a black car to the back of the ego vehicle. The IDs of these objects are <c1,CAM_FRONT,1080.0,520.0>, <c2,CAM_BACK,1080.0,520.0>, <c3,CAM_FRONT,1080.0,520.0>, and <c4,CAM_BACK,1080.0,520.0>. |
| llava-v1.6-mistral-7b-hf | In the current scene, the important objects to consider for future reasoning and driving decisions include:
1. The road ahead, which appears to be wet, indicating potential for hydroplaning or reduced traction.
2. The presence of vehicles, including cars and a motorcycle, which suggests that there is traffic and that drivers should be aware of other road users.
3. The streetlights, which provide illumination and help to identify the road's boundaries and potential hazards.
4. The pedestrian crossing, which indicates a place where pedestrians might be crossing the road.
5. The parked vehicles on the side of the road, which could obstruct the view of oncoming traffic or pedestrians.
6. The trees and foliage, which might limit visibility around curves or at intersections.
7. The presence of a cyclist, which requires drivers to be cautious and aware of cyclists sharing the road.
These objects are important for drivers to be aware of and consider when making driving decisions, especially in low-light conditions or at night. |
| 1 | Questionperception | What is the moving status of object <c1,CAM_FRONT,937.5,495.8>? Please select the correct answer from the following options: A. Back up. B. Turn right. C. Going ahead. D. Turn left. |
| GT Answer | C |
| InternVL2-2B | The moving status of object <c1,CAM_FRONT,937.5,495.8> is C. Going ahead. |
| Mini-InternVL2-2B-DA-DriveLM | C |
| llama_adapter_v2_multimodal7b | The moving status of object <c1,CAM_FRONT,937.5,495.8> is "Going ahead." |
| InternVL2-2b-LoRA-25k-drivelm | C |
| InternVL2-2b-LoRA-25k-drivelm-offline-redcircle-ctag-bkgd | C |
| InternVL2-2b-LoRA-25k_plus_DL-PL-10pct | C |
| InternVL2-2b-LoRA-300k-drivelm | C. Going ahead. |
| llava-v1.6-mistral-7b-hf | C. Going ahead. |
| 2 | Questionprediction | In this scenario, what object is most likely to consider <c3,CAM_FRONT,1001.2,556.5>? |
| GT Answer | <c2,CAM_FRONT,864.5,525.0>. |
| InternVL2-2B | In this scenario, the most likely object to consider `<c3,CAM_FRONT,1001.2,556.5>` is the car in the middle of the image. This is because the car is the only object that is clearly visible and is positioned in the center of the image, making it the most relevant object for the given criteria. |
| Mini-InternVL2-2B-DA-DriveLM | The ego vehicle. |
| llama_adapter_v2_multimodal7b | In this scenario, the most likely object to consider is the car, as it is the main subject of the image and is driving down the street. The other objects, such as the traffic light, streetlight, and trees, are also present in the image, but the car is the primary focus of the scene. |
| InternVL2-2b-LoRA-25k-drivelm | <c1,CAM_FRONT,1010.8,525.8>. |
| InternVL2-2b-LoRA-25k-drivelm-offline-redcircle-ctag-bkgd | <c2,CAM_FRONT,842.5,505.0>. |
| InternVL2-2b-LoRA-25k_plus_DL-PL-10pct | <c1,CAM_FRONT,1011.7,510.0>. |
| InternVL2-2b-LoRA-300k-drivelm | The ego vehicle. |
| llava-v1.6-mistral-7b-hf | The object most likely to consider the image with the label "<c3,CAM_FRONT,1001.2,556.5>" is a camera or a surveillance system that is positioned at the front of a vehicle, facing forward, and is likely to be mounted on the left side of the vehicle, as indicated by the coordinates. The image shows a view of the road ahead, which is consistent with the camera's position and orientation. |
| 3 | Questionprediction | Is <c1,CAM_FRONT,937.5,495.8> a traffic sign or a road barrier? |
| GT Answer | No. |
| InternVL2-2B | To determine if the coordinates <c1, CAM_FRONT, 937.5, 495.8> represent a traffic sign or a road barrier, let's break down the coordinates step by step:
1. **Coordinates Analysis:**
- **c1:** This is the starting point of the coordinates.
- **CAM_FRONT:** This is the direction of the camera, which is forward.
- **937.5:** This is the x-coordinate.
- **495.8:** This is the y-coordinate.
2. **Understanding the Coordinates:**
- The coordinates are given in a format that is typical for GPS coordinates. The format is usually (latitude, longitude).
3. **Interpreting the Coordinates:**
- **Latitude:** 937.5
- **Longitude:** 495.8
4. **Purpose of Coordinates:**
- **Traffic Sign:** A traffic sign typically has a specific shape and position on the road. The coordinates given here do not match the typical shape of a traffic sign.
- **Road Barrier:** A road barrier is usually a physical object that is placed on the road, such as a guardrail or a barrier post. The coordinates do not indicate any such object.
5. **Conclusion:**
- Given the coordinates are in a format that is typical for GPS coordinates and the coordinates do not match the shape of a typical traffic sign, it is reasonable to conclude that they represent a road barrier.
Therefore, the coordinates <c1, CAM_FRONT, 937.5, 495.8> are most likely a road barrier. |
| Mini-InternVL2-2B-DA-DriveLM | No. |
| llama_adapter_v2_multimodal7b | The image does not provide enough information to confidently determine whether the <c1,CAM_FRONT,937.5,495.8> is a traffic sign or a road barrier. It could be either, but without more context or a clearer view of the sign, it is not possible to definitively identify its purpose. |
| InternVL2-2b-LoRA-25k-drivelm | No. |
| InternVL2-2b-LoRA-25k-drivelm-offline-redcircle-ctag-bkgd | No. |
| InternVL2-2b-LoRA-25k_plus_DL-PL-10pct | No. |
| InternVL2-2b-LoRA-300k-drivelm | No. |
| llava-v1.6-mistral-7b-hf | The image you've provided appears to be a series of nighttime street scenes, likely taken from the perspective of a vehicle's rearview camera. The image labeled as <c1,CAM_FRONT,937.5,495.8> is showing the front view of a vehicle, which is not a traffic sign or a road barrier. It's a camera angle that captures the front of the vehicle and the road ahead. |
| 4 | Questionplanning | What actions could the ego vehicle take based on <c1,CAM_FRONT,937.5,495.8>? Why take this action and what's the probability? |
| GT Answer | The action is to keep going at the same speed. The reason is that there is no safety issue. The probability of this action is high. |
| InternVL2-2B | The ego vehicle, indicated by the camera's position and the label "CAM_FRONT_LEFT," is currently facing the left side of the road. The label "937.5,495.8" suggests that the camera's position is approximately 37.5 degrees to the left of the vehicle's current orientation.
Given this information, the ego vehicle could take the following actions:
1. **Turn Left**: The vehicle could turn left to navigate around the intersection or to avoid any obstacles ahead. The angle of 37.5 degrees suggests that the vehicle is not far from the left edge of the road, making a left turn feasible.
2. **Driving Straight**: The vehicle could continue straight if it is not approaching an intersection or if there are no immediate obstacles. The angle of 37.5 degrees is not too steep, making a straight drive possible.
3. **Driving Straight or Turn Right**: If the vehicle is approaching an intersection, it could either turn right or continue straight. The angle of 37.5 degrees is not too steep, making a right turn feasible.
**Probability**:
- **Turn Left**: The probability of turning left is high, as the vehicle is not too far from the left edge of the road. The angle is within a range that is generally considered safe for a left turn.
- **Driving Straight**: The probability of driving straight is also high, as the angle is not too steep and the vehicle is not approaching an intersection.
- **Driving Straight or Turn Right**: The probability of driving straight or turning right is moderate, as the angle is not too steep and the vehicle is not approaching an intersection.
In summary, the ego vehicle has a high probability of turning left or driving straight, depending on its current position and the direction it is facing. |
| Mini-InternVL2-2B-DA-DriveLM | The action is to keep going at the same speed. The reason is that there is no safety issue. The probability is high. |
| llama_adapter_v2_multimodal7b | Based on the given coordinates, the ego vehicle is at the position <c1,CAM_FRONT,937.5,495.8> in the image. The vehicle could take several actions, such as turning left, turning right, or staying in its current position. The probability of the vehicle turning left is 0.5, as the coordinates indicate that the vehicle is at the left side of the road. The probability of the vehicle turning right is also 0.5, as the coordinates indicate that the vehicle is at the right side of the road. The probability of the vehicle staying in its current position is 0.5, as the coordinates indicate that the vehicle is in the middle of the road. |
| InternVL2-2b-LoRA-25k-drivelm | The action is to keep going at the same speed. The reason is that there is no safety issue, and the probability is high. |
| InternVL2-2b-LoRA-25k-drivelm-offline-redcircle-ctag-bkgd | The action is to accelerate and go ahead. The reason is that the object has little reference value for the ego vehicle, with a high probability. |
| InternVL2-2b-LoRA-25k_plus_DL-PL-10pct | The action is to keep going at the same speed. The reason is that there is no safety issue. The probability is high. |
| InternVL2-2b-LoRA-300k-drivelm | The action is to keep going at the same speed. The reason is that there is no safety issue, and the probability is high. |
| llava-v1.6-mistral-7b-hf | The image you've provided appears to be a sequence of images showing a vehicle's perspective of a road at night, with the vehicle's position and orientation changing in each frame. The labels "CAM_FRONT_LEFT," "CAM_FRONT_RIGHT," "CAM_BACK_LEFT," and "CAM_BACK_RIGHT" suggest that these are the views from the front and back left and right cameras of the vehicle.
Based on the first image, the vehicle is at the front of the road, and it seems to be moving forward. The second image shows the vehicle's position slightly to the left, indicating that it has turned or is in the process of turning. The third image shows the vehicle's position further to the left, suggesting that it has continued to turn or is in a lane that is more to the left. The fourth image shows the vehicle's position at the back of the road, indicating that it has stopped or is in a position where it is not moving forward.
The actions the ego vehicle could take based on the first image are:
1. Continue straight: If the vehicle's driver intends to stay on the current path, they should continue straight.
2. Turn left: If the driver wants to turn left, they should start signaling and begin the turn.
3. Turn right: If the driver wants to turn right, they should start signaling and begin the turn.
4. Stop: If the driver needs to stop, they should apply the brakes.
The probability of each action depends on the driver's intended path and the current road conditions. If the driver intends to stay on the current path, the probability of continuing straight is high. If the driver wants to turn left or right, the probability of turning is moderate, and the probability of stopping is low. If the driver needs to stop, the probability of stopping is high.
The actual probability of each action would depend on the driver's intentions, the road conditions, and the presence of other vehicles or obstacles. |
| 5 | Questionplanning | What actions taken by the ego vehicle can lead to a collision with <c2,CAM_FRONT,864.5,525.0>? |
| GT Answer | Accelerating and going straight can lead to a collision with <c2,CAM_FRONT,864.5,525.0>. |
| InternVL2-2B | The ego vehicle is driving on a road at night, and it is approaching a curve. The image shows the ego vehicle's position relative to the road and the surrounding environment. Here are the actions that could lead to a collision with the specified point:
1. **Driving too fast**: The ego vehicle might be driving too fast, especially if it is approaching the curve at a high speed. This could cause the vehicle to lose control and collide with the road ahead.
2. **Not paying attention to the curve**: The ego vehicle might not be paying attention to the curve ahead, leading to a sudden change in speed or direction. This could result in a collision with the road or other vehicles.
3. **Not maintaining a safe following distance**: The ego vehicle might not be maintaining a sufficient distance from the vehicle ahead, especially if it is approaching the curve. This could lead to a collision if the vehicle ahead suddenly changes direction or speed.
4. **Not braking in time**: The ego vehicle might not have braked in time to avoid the curve, leading to a sudden stop or slowdown. This could cause the vehicle to lose control and collide with the road ahead.
5. **Not using the turn signal**: The ego vehicle might not be using the turn signal, which could make other drivers less aware of its intentions. This could lead to a collision if the vehicle turns suddenly.
6. **Not using the cruise control**: The ego vehicle might not be using the cruise control, which could result in a slower speed compared to the surrounding traffic. This could lead to a collision if the vehicle is not able to maintain a safe speed.
7. **Not following traffic rules**: The ego vehicle might not be following traffic rules, such as not stopping at a red light or not yielding to other vehicles. This could lead to a collision if the vehicle is not given enough time to react.
8. **Not being aware of the road conditions**: The ego vehicle might not be aware of the road conditions, such as wet or icy conditions, which could affect traction and control. This could lead to a collision if the vehicle loses traction.
9. **Not being prepared for sudden changes in traffic**: The ego vehicle might not be prepared for sudden changes in traffic, such as an unexpected stop or a sudden change in lane. This could lead to a collision if the vehicle is not able to react quickly enough.
10. **Not being prepared for the curve ahead**: The ego vehicle |
| Mini-InternVL2-2B-DA-DriveLM | No such action will lead to a collision. |
| llama_adapter_v2_multimodal7b | The ego vehicle is driving down the street, and it is approaching a red traffic light. If the ego vehicle continues to drive forward without stopping, it may collide with the <c2,CAM_FRONT,864.5,525.0> object, which is likely another vehicle or a pedestrian. The driver should stop the vehicle in accordance with the traffic light's signal to avoid a collision and ensure the safety of all road users. |
| InternVL2-2b-LoRA-25k-drivelm | Accelerating and going straight. |
| InternVL2-2b-LoRA-25k-drivelm-offline-redcircle-ctag-bkgd | Accelerating and going straight. |
| InternVL2-2b-LoRA-25k_plus_DL-PL-10pct | Accelerating and going straight. |
| InternVL2-2b-LoRA-300k-drivelm | Accelerating and going straight. |
| llava-v1.6-mistral-7b-hf | The image you've provided appears to be a series of snapshots from a vehicle's camera system, showing different views of the road ahead. The snapshots are labeled with various camera angles:
- CAM_FRONT_LEFT
- CAM_FRONT_RIGHT
- CAM_BACK_LEFT
- CAM_BACK_RIGHT
To avoid a collision with the vehicle labeled as <c2,CAM_FRONT,864.5,525.0>, the ego vehicle should take the following actions:
1. **Maintain a Safe Following Distance**: The ego vehicle should ensure it maintains a safe distance from the vehicle in front, especially when the road is wet or visibility is reduced.
2. **Stay in the Lane**: The ego vehicle should stay within its lane, as indicated by the CAM_FRONT_LEFT and CAM_FRONT_RIGHT snapshots.
3. **Monitor Surroundings**: The ego vehicle should be aware of its surroundings, including the vehicle in front and any potential obstacles or hazards.
4. **Use Turn Signals**: If the ego vehicle needs to change lanes or make a turn, it should use its turn signals to communicate its intentions to other drivers.
5. **Slow Down**: If the road conditions are poor or if the vehicle in front is driving too fast for the conditions, the ego vehicle should slow down to match the speed of the vehicle in front.
6. **Be Prepared for Sudden Stops**: The ego vehicle should be prepared to stop suddenly if the vehicle in front stops or if there is an obstacle in the road.
7. **Use Headlights**: If the road is dark or visibility is reduced, the ego vehicle should turn on its headlights to improve visibility.
8. **Avoid Distractions**: The driver should avoid any distractions, such as using a mobile phone or engaging in activities that could divert their attention from the road.
9. **Stay Alert**: The driver should stay alert and focused on the road, especially when the road is wet or visibility is reduced.
10. **Follow Traffic Rules**: The driver should follow all traffic rules and regulations, including speed limits, lane markings, and right-of-way rules.
By following these guidelines, the ego vehicle can reduce the risk |
| 6 | Questionplanning | In this scenario, what are safe actions to take for the ego vehicle? |
| GT Answer | Brake gently to a stop, change to the right lane, and decelerate gradually without braking. |
| InternVL2-2B | Based on the image, the safe actions for the ego vehicle would include maintaining a safe following distance from the vehicle in front, as indicated by the "CAM_FRONT_LEFT" and "CAM_FRONT" labels. The vehicle should also be prepared to react to any sudden movements or changes in traffic, as suggested by the "CAM_FRONT_LEFT" and "CAM_FRONT" labels. Additionally, the vehicle should be cautious of the vehicle in front, as indicated by the "CAM_FRONT_LEFT" and "CAM_FRONT" labels, and should not attempt to overtake or pass when the vehicle in front is stationary or moving slowly. The "CAM_BACK_LEFT" and "CAM_BACK" labels suggest that the vehicle should be aware of the vehicle in the opposite direction and should maintain a safe distance to avoid collisions. The "CAM_BACK_RIGHT" label indicates that the vehicle should be prepared to react to any changes in traffic on the opposite side of the road. |
| Mini-InternVL2-2B-DA-DriveLM | Slightly offset to the left, slightly offset to the right, and keep going at the same speed are safe actions to take for the ego vehicle in this scenario. |
| llama_adapter_v2_multimodal7b | In this scenario, the ego vehicle is driving down a street at night with multiple traffic lights and other vehicles. To ensure safety, the driver should follow these actions:1. Obey traffic signals: The driver should pay close attention to the traffic lights and follow the signals, such as stopping at red lights and proceeding through green lights.2. Maintain a safe distance: The driver should keep a safe distance from other vehicles, especially those in front, to avoid collisions.3. Use headlights: The driver should turn on their headlights to improve visibility and signal their presence to other drivers.4. Be cautious of pedestrians: The driver should be aware of pedestrians crossing the street or walking near the road, and exercise caution when approaching them.5. Stay alert: The driver should remain vigilant and attentive to their surroundings, including other vehicles, pedestrians, and potential obstacles.By following these safe actions, the driver can minimize the risk of accidents and ensure a smooth and secure driving experience. |
| InternVL2-2b-LoRA-25k-drivelm | Keep going at the same speed, accelerate, and proceed. |
| InternVL2-2b-LoRA-25k-drivelm-offline-redcircle-ctag-bkgd | Keep going at the same speed, decelerate gradually without braking. |
| InternVL2-2b-LoRA-25k_plus_DL-PL-10pct | Keep going at the same speed, decelerate gradually without braking. |
| InternVL2-2b-LoRA-300k-drivelm | Keep going at the same speed, decelerate gradually without braking. |
| llava-v1.6-mistral-7b-hf | In the given scenario, the ego vehicle is driving on a road at night. Here are some safe actions to take:
1. **Turn on Headlights**: The vehicle's headlights should be turned on to improve visibility and signal to other drivers that the vehicle is in operation.
2. **Slow Down**: Nighttime driving conditions can be more challenging due to reduced visibility. Slowing down allows for more reaction time and better control of the vehicle.
3. **Maintain a Safe Following Distance**: Keeping a safe distance from the vehicle in front allows for more time to react to any sudden changes in traffic.
4. **Use Indicators**: When changing lanes or making turns, use the vehicle's indicators to signal intentions to other drivers.
5. **Stay Alert**: Nighttime driving can be more tiring due to the lack of natural light. Staying alert and focused on the road is crucial.
6. **Avoid Distractions**: Keep the vehicle's interior lights to a minimum to avoid glare and distractions.
7. **Be Prepared for Pedestrians**: Nighttime can be more dangerous for pedestrians, so be extra vigilant for people crossing the road or walking along the sidewalk.
8. **Watch for Animals**: In some areas, animals may be more active at night. Be prepared to react if an animal suddenly appears on the road.
9. **Use High Beam Headlights**: If the road is clear and there are no oncoming vehicles, use high beam headlights to improve visibility.
10. **Regularly Check and Maintain the Vehicle**: Ensure that the vehicle's headlights, taillights, and other lights are functioning properly. Regular maintenance can help prevent breakdowns at night.
Remember, driving at night requires extra caution and attention to the road and surroundings. |
| 7 | Questionbehavior | Predict the behavior of the ego vehicle. Please select the correct answer from the following options: A. The ego vehicle is slightly steering to the right. The ego vehicle is driving fast. B. The ego vehicle is slightly steering to the right. The ego vehicle is driving very fast. C. The ego vehicle is steering to the right. The ego vehicle is driving slowly. D. The ego vehicle is going straight. The ego vehicle is driving with normal speed. |
| GT Answer | B |
| InternVL2-2B | Based on the image, the ego vehicle is driving straight. The image shows the ego vehicle on a straight road with no visible signs of steering or changing lanes. The other options are not supported by the visual evidence in the image. |
| Mini-InternVL2-2B-DA-DriveLM | D |
| llama_adapter_v2_multimodal7b | The ego vehicle is steering to the right, and it is driving fast. |
| InternVL2-2b-LoRA-25k-drivelm | D |
| InternVL2-2b-LoRA-25k-drivelm-offline-redcircle-ctag-bkgd | A |
| InternVL2-2b-LoRA-25k_plus_DL-PL-10pct | D |
| InternVL2-2b-LoRA-300k-drivelm | B. The ego vehicle is slightly steering to the right. The ego vehicle is driving very fast. |
| llava-v1.6-mistral-7b-hf | A. The ego vehicle is slightly steering to the right. The ego vehicle is driving fast. |
