I have been debugging slow loading on our CDR pages for the last few days now.
One issue that we have encountered is that currently as the page loads, it checks the filesystem for each file one at a time. In our case, we move recordings to object storage after 1 week, so each time we check for a file it passes api calls which take over 1 second each to return a result. This causes this page to not load at all for us in many cases.
Regardless, this current method is unnecessarily I/O intensive and really page load is probably not the time to be checking for each file one by one.
So this PR is the simplest solution - remove the check entirely. I would contend that the administrator should remove the record_path from the database if the file was removed so this should be acceptable.
This solves this particular issue for us, but would need feedback from others if not checking for files makes sense.
Track whether or not a message was actually left in the voicemail box. Previously we only knew that voicemail answered, now we know whether the caller left a message.
Callers who didn't leave a message now show up in the "Cancelled" call filter in xml_cdr.php
Bonus: Fixed a bug with the originating_leg_uuid that was breaking extension summary from a previous commit and some other minor bugs/typos.
Excluded cc_side = agent calls from being marked as missed_call = true
Fixed the previous performance issue with adding the cc_side != 'agent' to the SQL and removed its filter from the rendering loop for the xml_cdr.
It is redundant to filter out LOSE_RACE when originating_leg_uuid is also filtered, there is an overlap where every call with LOSE_RACE also has an originating_leg.
For some unexplained reason, including the `"and cc_side != 'agent'` in the WHERE tanks the query performance from seconds to minutes on Postgres 9.4. It runs great on Postgresql 13. Reverting to the "blank content while writing the page content" approach for this value unless I can find the source of the problem. - Oh, also removed an unnecessary condition that prevents you from filtering by LOSE_RACE.
Same thing as in the xml_cdr.php page display. If the call is answered instantly, less than a second, then the difference is 0s, and the 0s is a visual indicator that the call was answered, it just took less than a second. Calls that didn't get answered have a large negative number stored in the TTA field, 0 is an answered call.
Re-implemented the commented out "Failed" call status SQL filter. It was no different than leaving the "Call status" search box empty. Removed the send_refuse restriction.
If the TTA is 0 because the call was answered in less than a second (so that the answer_epoch and start_epoch are in the same second), it would not display the TTA at all. This is safe to include 0 because "unanswered" calls are going to have a TTA that is is the negative value of the start time, significantly lower than 0.
Changes
--------
- Improve CDR Import Logic so that missed_call column is more accurate to the "missed" status. It would previously mark unanswered outbound calls as "missed". These are their own category of call.
- Don't mark the CDRs of the "legs" of an Enterprise Ring Group call as missed, only the originating_leg will be marked (one missed call per call) - We could also just "skip" importing these call legs. Simultaneous ring groups don't have these duplicated CDRs for every ringing phone. The "Skip" approach might make most of the rest of this work irrelevant.
- Create `originating_leg_uuid` column in v_xml_cdr and import it into the database during CDR imports so it is available for filtering Enterprise Ring Group calls out of CDRs and reports.
- Move logic that hides the agent leg of CC calls, LOSE_RACE calls, and the Enterprise Leg hiding code from xml_cdr.php into xml_cdr_inc.php into the SQL query WHERE clause so the CDR page looks more consistent. The logic is the same, but these calls are now excluded from the query result entirely instead of having to "skip" rendering them in the list on the xml_cdr.php page.
- Improved CDR statistics page to use the missed_call variable instead of relying upon billsec and answer_stamp/answer_epoch. Added the same logic as the xml_cdr pages to the query so it excludes enterprise ring group call legs.
- Laid the query groundwork in xml_cdr_statistics to report on Average TTA (No UI changes yet to include that statistic)
Retroactive Changes
---------------------
There are a few changes going back in time to bring everything in line with this better reporting accuracy:
- If you want the populated the `originating_leg_uuid column` in `v_xml_cdr`, it will rely upon having the `json` column and not having deleted the data from it like I know some people do for space saving.
- If you don't have the json column, you are mostly out of luck for hiding the duplicate legs of Enterprise ring group calls. It might be possible, but it isn't going to be easy.
- On Newer Versions of postgres, this works:
```
UPDATE v_xml_cdr SET originating_leg_uuid = (json->'variables'->>'originating_leg_uuid')::uuid WHERE json->'variables'->>'originating_leg_uuid' IS NOT NULL;
```
- For some reason on postgres 9.4, I had to UPDATE every single record because I couldn't get it to allow the json syntax properly after the WHERE. This is fine, it doesn't change the end result it just means it has to run the UPDATE on every record, which will take a while
```
UPDATE v_xml_cdr SET originating_leg_uuid = (json->'variables'->>'originating_leg_uuid')::uuid;
```
- To remove the `missed_call = true` on all your previous outbound records so that they don't show up when you filter on missed (outbound unanswered calls can be accurately listed with TTA max 0 and direction outbound)
```
UPDATE v_xml_cdr SET missed_call = false WHERE direction = 'outbound' AND missed_call = true;
```
In some cases, this seems to speed up xml_cdr loading times by ~10x.
It appears one big cause of this is the final ORDER_BY statements are very slow in PostgreSQL for timestamp fields. Ordering by start_epoch field improves query execution time in a dramatic way and should result in the same ordering.
* Remove CDR stattistics from user group menu
users don't have CDR statistics vier permission no need to have it in the menu
* remove status from user group menu
No need to have it in the default menu for the user group
agree
emaktech
demonspork
agree
FusionPBX
fusionate
Nate