Tutorial How to enable real VOD load balancing without having .mkv files locally

[iptvman367]

Hello everyone,

Please forgive me for my English if I made some typos.

I am new to the world of IPTV but I have more than 25 years of experience IT fields.

I would like to share with you this tutorial that I've found by myself in order to contribute to the community in return for the useful information that I was able to find thanks to this forum in recent weeks during the launch of my iptv service.

The situation is as follows, I have around 60TB of films & series. I searched desperately for days how to achieve a real Load Balancing of my video files but without needing to have a physical copy of the videos files on each server but only using symlinks.
After several sleepless nights I've finally found a solution that works perfectly, and I will provide you with all the details below.

To view the content, you need to Sign In or Register.

Maybe I was p**** off for nothing and there are simpler solutions but not having found one, I'm sharing my own solution which is fully tested and functional.

Hope this will be helpfull for everyone

Thank you and feel free to comment if you need any information.

ENJOY
Iptvman

 

[Youmeirean]

Hello everyone,

Please forgive me for my English if I made some typos.

I am new to the world of IPTV but I have more than 25 years of experience IT fields.

I would like to share with you this tutorial that I've found by myself in order to contribute to the community in return for the useful information that I was able to find thanks to this forum in recent weeks during the launch of my iptv service.

...........

Thank you and feel free to comment if you need any information.

ENJOY
Iptvman

great tuto mate well done will this work in my example lb1=600tb of files im maxing out the port so i need to load balance if i add a lb2 example how can i sysmlink so it uses both lbs ports thanks in advance

 

[iptvman367]

great tuto mate well done will this work in my example lb1=600tb of files im maxing out the port so i need to load balance if i add a lb2 example how can i sysmlink so it uses both lbs ports thanks in advance

Hello,
Thank you
Just to be sure, the 600tb of files are locally stocked on your LB1 ?
If yes, i have another solution that fit for you as I also have tested it initially.
LB1<----------------->LB2, you have to configure vpn site-to-site between your LBs. (I'm using openVPN)
Then on your LB1, install samba and define the shared folder where are located all your movies files.
On LB2, configure samba to access the shared folder of LB1 using it's private vpn IP.
Then you can follow my tutorial and on mysql step, be sure to replace the ID with ids of your LB1 and LB2

But use this technique only if you have enough bandwith on your LB1.

 

[Youmeirean]

Hello,
Thank you
Just to be sure, the 600tb of files are locally stocked on your LB1 ?
If yes, i have another solution that fit for you as I also have tested it initially.
LB1<----------------->LB2, you have to configure vpn site-to-site between your LBs. (I'm using openVPN)
Then on your LB1, install samba and define the shared folder where are located all your movies files.
On LB2, configure samba to access the shared folder of LB1 using it's private vpn IP.
Then you can follow my tutorial and on mysql step, be sure to replace the ID with ids of your LB1 and LB2

But use this technique only if you have enough bandwith on your LB1.

when trying to copy symlinks to lb2 i get this error
/usr/bin/rsync: Argument list too long

 

[iptvman367]

when trying to copy symlinks to lb2 i get this error
/usr/bin/rsync: Argument list too long

This mean you try to sync too many files at once, try to split it into smaller chunks.
you can use --files-from to divide it
--files-from=/path/to/filelist.txt
--files-from=/path/to/filelist0.txt
--files-from=/path/to/filelist1.txt
--files-from=/path/to/filelis2.txt
...
in each filelistxx.txt you insert a list of symlinks mkv files to copy

 

[Youmeirean]

This mean you try to sync too many files at once, try to split it into smaller chunks.
you can use --files-from to divide it
--files-from=/path/to/filelist.txt
--files-from=/path/to/filelist0.txt
--files-from=/path/to/filelist1.txt
--files-from=/path/to/filelis2.txt
...
in each filelistxx.txt you insert a list of symlinks mkv files to copy

copied so the database part instead of main im guessing i use lb1 lb2 not 1 and 7

 

[iptvman367]

copied so the database part instead of main im guessing i use lb1 lb2 not 1 and 7

Yes but first be sure to backup your database.
then go to manage servers and check ids of your current Lbs

 

[extacy]

Can you recommend a place where this type of service can be obtained at an affordable price?
Single 50 TB or 2-3 parts. Approximately 100 TB of traffic per month is sufficient.

 

[iptvman367]

Can you recommend a place where this type of service can be obtained at an affordable price?
Single 50 TB or 2-3 parts. Approximately 100 TB of traffic per month is sufficient.

Hi
You can look per example at B2, e2, Mega…
It’s better to divide into multi buckets of 15/20 TB each and dispatch them in multi location, but depends also where your customers are located.

 

[Youmeirean]

This mean you try to sync too many files at once, try to split it into smaller chunks.
you can use --files-from to divide it
--files-from=/path/to/filelist.txt
--files-from=/path/to/filelist0.txt
--files-from=/path/to/filelist1.txt
--files-from=/path/to/filelis2.txt
...
in each filelistxx.txt you insert a list of symlinks mkv files to copy

me again lol question how are you creating these txt files surly not typing them out i have over 4 million symlinks so typing each would take years

 

[iptvman367]

me again lol question how are you creating these txt files surly not typing them out i have over 4 million symlinks so typing each would take years

First tell me how many files does contains your vod folder by using this command
ls | wc

 

[Youmeirean]

First tell me how many files does contains your vod folder by using this command
ls | wc

441553

First tell me how many files does contains your vod folder by using this command
ls | wc

653362 653362 8944226 this is the output

 

[iptvman367]

653362 653362 8944226 this is the output

Ok this is a lot lol
First enter this command to generate the file in the source directory

To view the content, you need to Sign In or Register.

 

[Youmeirean]

Ok this is a lot lol
First enter this command to generate the file in the source directory

*** Hidden text: cannot be quoted. ***

so it seems it dont work for local vod iv copied symlinks to lb iv mounted lb1 to lb2 example to /home/tvshows so its located in the same place on lb2 as lb1 but seems its copying the whole show to lb2

 

[iptvman367]

so it seems it dont work for local vod iv copied symlinks to lb iv mounted lb1 to lb2 example to /home/tvshows so its located in the same place on lb2 as lb1 but seems its copying the whole show to lb2

You must have mistaken somewhere, check one of the symlink and be sure it's the same on both LB and the access path must be the same.
Also the mkv file it's accessible from symlink ? Have you checked permission on file ?
Also did you add all the necessary on mysql database ?
you can send me screenshot on PM if you want we gonna look together

Also don't forget to check firewall rules

 

[Youmeirean]

You must have mistaken somewhere, check one of the symlink and be sure it's the same on both LB and the access path must be the same.
Also the mkv file it's accessible from symlink ? Have you checked permission on file ?
Also did you add all the necessary on mysql database ?
you can send me screenshot on PM if you want we gonna look together

Also don't forget to check firewall rules

Drop me a DM bud if that’s ok

 

[Moutamared]

but this way the movies played as a stream not a VOD, right ?

 

[iptvman367]

but this way the movies played as a stream not a VOD, right ?

Absolutely not its always remain VOD