For ability to open the booking form in popup window (in paid versions of Booking Calendar), please follow these steps:
1) Create new page (or edit existing page) and add the "Custom HTML block" to the page.
Then insert the code similar to this:

<script src="https://server.com/wp-content/plugins/booking/assets/libs/ui/_out/dropdown_modal.js"></script>
<a  href="javascript:void(0)" onclick="javascript:jQuery('.bookingFormModal').wpbc_my_modal('show');"
    class="wp-element-button">Booking</a>
[booking resource_id=1]

Where [booking type=1] - shortcode for your booking form. Please check more here.

Also you need to define correct URL to dropdown_modal.js script.

<script src="https://server.com/wp-content/plugins/booking/assets/libs/ui/_out/dropdown_modal.js"></script>

You need to replace server.com to your website DNS. And if you are using some paid version of Booking Calendar, then you need to replace /booking/ to correct Booking Calendar Folder. Usually it's similar to this: /booking.bs.10.0/

2) Now open the WP Booking Calendar > Settings > Booking Form page and configure the booking form similar to this:

<div class="wpdevelop">
    <div class="modal bookingFormModal" tabindex="-1" role="dialog" style="display:none;z-index:9999999;">
        <div class="modal-dialog modal-lg" role="document">
            <div class="modal-content">
              <div class="modal-header"><button type="button" class="close" data-dismiss="modal" aria-label="Close"><span aria-hidden="true">×</span></button><h4 class="modal-title">Booking Form</h4></div>
              <div class="modal-body">
<!-- Start Booking form configuration -->				  
 				  
[calendar] 
<div class="standard-form"> 
     <p>First Name (required):<br />[text* name] </p> 
     <p>Last Name (required):<br />[text* secondname] </p> 
     <p>Email (required):<br />[email* email] </p>   
     <p>Phone:<br />[text phone] </p> 
     <p>Adults:<br />[select visitors "1" "2" "3" "4"]</p> 
     <p>Children:<br />[select children "0" "1" "2" "3"]</p> 
     <p>Details:<br /> [textarea details] </p> 
     <p>[checkbox* term_and_condition use_label_element "I Accept term and conditions"] </p>
     <p>[captcha]</p> 
     <p>[submit class:btn "Send"]</p> 
</div>
				  			  
 <!-- End configuration  of the booking form -->
             </div>
            </div>
        </div>
    </div>
</div>
<style type="text/css"> .wpdevelop .modal { overflow-y: scroll; } </style>

3) The result will be like this:

Such configuration possible in the Booking Calendar updates 9.8 or newer.